General issues regarding F2FS rootfs partition

[ToastBucket]

You can also change your uEnv.txt to have the eMMC bootloader boot from different media based on the state of a GPIO (and hopefully soon from an onboard keyboard key). This way you keep your bootloader on the eMMC and you can have it boot the OS on the uSD by default. Then you can boot from the eMMC OS by, say holding “r” for recovery.

 

[God Ginrai]

I disagree. It is good for dualbooting too. Emmc is the main OS, and microsd could be a testing os, or maybe Android, bsd, etc. Thus, if it autodetects, and boots, that negates it. Unless you have a shortcut to bypass, then we are back into the same boat.

Sent from my H3123 using Tapatalk

If the hardware autodetects and chooses, the behavior is simpler, but less flexible. This gives more options. Plus, as the bootloader is open source and easily customizable without building it or reflashing it, you can always put the bootloader on the eMMC and have it boot whatever medium you want. You could also have it select which medium based on which key is pressed at boot (once I write a UBoot driver for the keyboard).

I definitely agree that it is kind of annoying and I personally choose not to use it, but there are some people that are a fan of this setup.

You could easily have this functionality still be just as flexible by making the L1 just disable the automatic selection. That way, the functionality works best for the majority of users using the microsd while still allowing that flexibility of choice.

You can also change your uEnv.txt to have the eMMC bootloader boot from different media based on the state of a GPIO (and hopefully soon from an onboard keyboard key). This way you keep your bootloader on the eMMC and you can have it boot the OS on the uSD by default. Then you can boot from the eMMC OS by, say holding “r” for recovery.

Yes, but this leads to the issue that your boot configuration is reliant on the eMMC not dying. The whole reason we asked for the microSD alternative was so that we had an option for the OS in the case that the eMMC died on us. (or alternatively so that we can avoid such a case ever happening by not using it) If the eMMC dies, you're back to being forced to hold L1 again because there is no automatic selection when the card slot is filled.

-God Ginrai

 

[levi]

I'm not 100% sure, but I think the flash inside the CPU checks the left SD slot for anything bootable before reading the eMMC. So if your eMMC becomes unwritable, and is stuck with bad data, you can still stick a card in the left slot to recover the system. That's less ideal, as it means your Pyra has effectively lost a big SD slot that's externally accessible, but luckily you still have the right slot, and I believe it's possible to put an OS and all of your dbps on one card in the left slot.

 

[ToastBucket]

You could easily have this functionality still be just as flexible by making the L1 just disable the automatic selection. That way, the functionality works best for the majority of users using the microsd while still allowing that flexibility of choice.

I agree that an automatic detection method based on the card detect line of the uSD would be clever, but the core issue still remains. The behavior is just shifted in your favor. If a user wishes to boot off the eMMC and use a uSD for storage or rootfs, the issue remains but in reverse. The hardware autodetect will select the uSD and try to boot off that and fail. If we disable autodetection, which device gets selected? The existing method is simpler to the end user. Do nothing, eMMC is selected. Hold L1, uSD is selected. This hardware logic could be reversed to select the uSD by default, but as stated before the core issue remains.

For those worried about the eMMC and don't want to use it at all, the best method is to store your bootloader and /boot partition on the Left SD and use the uSD or Left SD for your OS. Everything involved with booting (SPL, U-Boot, and /boot partition) is only read, except at the time of updating the bootloader or kernel package. You can also use the eMMC to store only this, writes to the device will be very infrequent with this method and it will be a long time until it dies. This is rather similar to how I have mine configured.

A wiki page on how to customize the boot process will be created soon. The bootloader is very close to stabilization.

I'm not 100% sure, but I think the flash inside the CPU checks the left SD slot for anything bootable before reading the eMMC. So if your eMMC becomes unwritable, and is stuck with bad data, you can still stick a card in the left slot to recover the system. That's less ideal, as it means your Pyra has effectively lost a big SD slot that's externally accessible, but luckily you still have the right slot, and I believe it's possible to put an OS and all of your dbps on one card in the left slot.

Yes the Left SD slot is the first place the bootROM checks for a bootloader.

 

[theredbaron]

I actually plan to, once we have a 'stable' OS and bootloader, do just the opposite of what you describe.

The internal eMMC will have a full OS with utilities and whatnot on it.

For regular use, I'll hold a shoulder button and boot into an internal 1TB microSDXC card. I have done some initial testing of this (using a 64GB microSDXC card) and it works.

For right now, though, nearly everyone is booting from the left SDXC slot simply for convenience of swapping out the OS frequently.

From what I was able to tell on my non-scientific-seat-of-pants-mess-around-experiment, the internal microSDXC boots faster than the left SDXC slot. The eMMC is of course faster than both, but not by enough to really make a material difference in usage. I'll simply be sacrificing a small bit of speed for a boatload more active internal storage. Having to press a shoulder button during boot to get that additional storage is nothing.

Is the micro fast enough for that? I always look at os on cards as a backup due to speed.

Sent from my H3123 using Tapatalk

 

[levi]

Depends what you consider slow. I'm currently booting most of my x86 OS off an sd card taped into the USB2 internal card reader, and it might slow down boot up by a small percentage, and slow down the loading of programmes, but I don't run swap here, so once they're loaded they run just fine.

 

[Grench]

Is the micro fast enough for that? I always look at os on cards as a backup due to speed.

Yes. It has been months and I'm doing some recall here that could be subject to mental error, but to my recollection:
Booting from eMMC took about 35 seconds to password prompt.
Booting from left SDXC took about 45 seconds to password prompt.
Booting from the internal microSDXC took about 42 seconds to password prompt.

I had used the same 64GB microSDXC card (with adapter for SDXC). No stopwatch was used, just me watching a clock. I didn't bother doing anything formal because at the time there wasn't anything 'formal' about the OS install. Some day when we have the next 'release candidate' OS, I might be able to run that with a bit more formality.

Remember that the internal microSDXC is actually running on the eMMC controller which seemed to me to be marginally faster than the left SDXC slot. Yes, as expected it IS considerably slower than the eMMC. After all the microSDXC is only using half the data lines on the eMMC. But - it is not 'slow' by any stretch. Seat of the pants it feels like the difference between 70 MB/s and 120 MB/s, The Pandora on a good day could pull 18 MB/s on the SDXC slot.

Some day, months after general release, I would like to see us being able to actively switch the MUX on the eMMC / microSDXC in session. I figure under those use cases where copying a file from eMMC to microSDXC without caching each one would get knocked down to about 25% of normal performance. But, with how Linux caches writes, it won't really be much of a performance hit unless you're copying HUGE amounts of data between the two - which isn't a valid 'regular' use case. Just be sure to put a root level sync command somewhere in the lid close / hibernate / sleep & shutdown process. Then I would want to boot from the eMMC and have /home and /usr on the microSDXC. But - that cannot be a 'shipping default'.

 

[ToastBucket]

Depends what you consider slow. I'm currently booting most of my x86 OS off an sd card taped into the USB2 internal card reader, and it might slow down boot up by a small percentage, and slow down the loading of programmes, but I don't run swap here, so once they're loaded they run just fine.

I would rip my hair out.

 

[theredbaron]

Yes. It has been months and I'm doing some recall here that could be subject to mental error, but to my recollection:
Booting from eMMC took about 35 seconds to password prompt.
Booting from left SDXC took about 45 seconds to password prompt.
Booting from the internal microSDXC took about 42 seconds to password prompt.

I had used the same 64GB microSDXC card (with adapter for SDXC). No stopwatch was used, just me watching a clock. I didn't bother doing anything formal because at the time there wasn't anything 'formal' about the OS install. Some day when we have the next 'release candidate' OS, I might be able to run that with a bit more formality.

Remember that the internal microSDXC is actually running on the eMMC controller which seemed to me to be marginally faster than the left SDXC slot. Yes, as expected it IS considerably slower than the eMMC. After all the microSDXC is only using half the data lines on the eMMC. But - it is not 'slow' by any stretch. Seat of the pants it feels like the difference between 70 MB/s and 120 MB/s, The Pandora on a good day could pull 18 MB/s on the SDXC slot.

Some day, months after general release, I would like to see us being able to actively switch the MUX on the eMMC / microSDXC in session. I figure under those use cases where copying a file from eMMC to microSDXC without caching each one would get knocked down to about 25% of normal performance. But, with how Linux caches writes, it won't really be much of a performance hit unless you're copying HUGE amounts of data between the two - which isn't a valid 'regular' use case. Just be sure to put a root level sync command somewhere in the lid close / hibernate / sleep & shutdown process. Then I would want to boot from the eMMC and have /home and /usr on the microSDXC. But - that cannot be a 'shipping default'.

Dang man, very usable. Last time I 'tried" running an os off an sd card, took near 2 min to boot. Sub 1 min is doable, especially considering the huge boost in capacity over the emmc.

Sent from my H3123 using Tapatalk

 

[levi]

Jesus, 32 GB for an OS not including most of your home directory? My 32-bit OS here currently fits into 7GB for /usr, 4GB for /var (could be bigger, but then I run archlinux here which updates more often than debian), and less than half a gig for everything else. 32 gig is twice as much as I can use.

 

[God Ginrai]

I agree that an automatic detection method based on the card detect line of the uSD would be clever, but the core issue still remains. The behavior is just shifted in your favor. If a user wishes to boot off the eMMC and use a uSD for storage or rootfs, the issue remains but in reverse. The hardware autodetect will select the uSD and try to boot off that and fail. If we disable autodetection, which device gets selected? The existing method is simpler to the end user. Do nothing, eMMC is selected. Hold L1, uSD is selected. This hardware logic could be reversed to select the uSD by default, but as stated before the core issue remains.

For those worried about the eMMC and don't want to use it at all, the best method is to store your bootloader and /boot partition on the Left SD and use the uSD or Left SD for your OS. Everything involved with booting (SPL, U-Boot, and /boot partition) is only read, except at the time of updating the bootloader or kernel package. You can also use the eMMC to store only this, writes to the device will be very infrequent with this method and it will be a long time until it dies. This is rather similar to how I have mine configured.

A wiki page on how to customize the boot process will be created soon. The bootloader is very close to stabilization.


Yes the Left SD slot is the first place the bootROM checks for a bootloader.

Yea, I think reversing it is what I actually meant by disabling the automatic selection. The difference in the implementation would be that when the eMMC dies, the user isn't stuck having to hold a button all the time. That's a big difference. And your "best method" is exactly the situation that we wanted to avoid by having the microSD in the first place. (We didn't want to have to specially format our normal SDs and waste space on them for boot)

-God Ginrai

 

[Grench]

Yea, I think reversing it is what I actually meant by disabling the automatic selection. The difference in the implementation would be that when the eMMC dies, the user isn't stuck having to hold a button all the time. That's a big difference. And your "best method" is exactly the situation that we wanted to avoid by having the microSD in the first place. (We didn't want to have to specially format our normal SDs and waste space on them for boot)

-God Ginrai

You said, "when the eMMC dies" as if that is an inevitable default.

The odds of the eMMC dying in 'normal use' before all of us die of old age is pretty slim.

 

[Phlyra]

You said, "when the eMMC dies" as if that is an inevitable default.

The odds of the eMMC dying in 'normal use' before all of us die of old age is pretty slim.

Any chance of being able to add a PCIe SSD in future?

 

[spud42]

there would be no room inside the Pyra to put one even if the SOC had the needed interface. you could buy a M.2 SSD to USB enclosure but what would be the point?

 

[Phlyra]

there would be no room inside the Pyra to put one even if the SOC had the needed interface. you could buy a M.2 SSD to USB enclosure but what would be the point?

Oh? I thought they were really thin - is the space inside that constrained?!
Nah I’ll just use a USB SSD like the Corsair Flash Voyager GTX.

 

[ToastBucket]

Yea, I think reversing it is what I actually meant by disabling the automatic selection. The difference in the implementation would be that when the eMMC dies, the user isn't stuck having to hold a button all the time. That's a big difference. And your "best method" is exactly the situation that we wanted to avoid by having the microSD in the first place. (We didn't want to have to specially format our normal SDs and waste space on them for boot)

-God Ginrai

Like Grench said, it will be a long long time before the eMMC dies. Everyone seems to be very worried about this. Somewhat reasonable, but I think people are overly fearful of this. In my opinion, the performance benefits of the eMMC over the uSD (which are quite significant) outweigh the potential of it dying in maybe 10 years. Not to mention if it dies, the device is still usable in 2 other manners (uSD and left SD)

Most of the fear circulating eMMC wearout comes from their increased usage in Electric vehicles. People now worry about eMMCs dying because they are in vehicles which are expected to last for 10+ years. The cost of the $30 eMMC dying in a vehicle ruins the usability (and potentially safety) of a $50k vehicle rather than an $800 device.

One thing also to note, the high eMMC usage in devices these days is typically in phones. Most of the testing and metrics is gathered based on typical AOSP usage. Phone apps (and AOSP) are VERY data hungry compared to what is typically running on the Pyra. Most read/writes to files on Linux are going direct into the kernel rather than to the filesystem, so the eMMC usage on the Pyra is significantly lower under standard application than on say Android.

According to the AOSP team, Android apps will write up to or more than 10GB of data per day on the eMMC. This is massive. The only application I can see really writing large amounts of data frequently on the Pyra right now is a web browser caching pages and data, and that is not likely to be on the same order unless you’re visiting thousands of new data hungry pages every day. It would be nice to gather some disk usage metrics on the Pyra to set people at ease. Anyone with a prototype interested in doing some tests?

https://source.android.com/devices/tech/perf/flash-wear

In terms of wasting space, the bootloader is < 1MB and the kernel and related files are < 50MB. Right now we keep this on a 256MB partition so there is plenty of room for upgrades and multiple kernels etc, but it’s not necessary to be this large. With the cost of SD cards and such these days, is that really a huge deal? I picked up a 128GB card (which I use for my home partition) for like $30 and the filesystem and partition map overhead alone was greater than that.

Oh? I thought they were really thin - is the space inside that constrained?!
Nah I’ll just use a USB SSD like the Corsair Flash Voyager GTX.

Yes. Not much space inside. Those SSDs are large and there’s a lot going on inside.

 

[Phlyra]

Like Grench said, it will be a long long time before the eMMC dies. Everyone seems to be very worried about this. Somewhat reasonable, but I think people are overly fearful of this. In my opinion, the performance benefits of the eMMC over the uSD (which are quite significant) outweigh the potential of it dying in maybe 10 years. Not to mention if it dies, the device is still usable in 2 other manners (uSD and left SD)

Most of the fear circulating eMMC wearout comes from their increased usage in Electric vehicles. People now worry about eMMCs dying because they are in vehicles which are expected to last for 10+ years. The cost of the $30 eMMC dying in a vehicle ruins the usability (and potentially safety) of a $50k vehicle rather than an $800 device.

One thing also to note, the high eMMC usage in devices these days is typically in phones. Most of the testing and metrics is gathered based on typical AOSP usage. Phone apps (and AOSP) are VERY data hungry compared to what is typically running on the Pyra. Most read/writes to files on Linux are going direct into the kernel rather than to the filesystem, so the eMMC usage on the Pyra is significantly lower under standard application than on say Android.

According to the AOSP team, Android apps will write up to or more than 10GB of data per day on the eMMC. This is massive. The only application I can see really writing large amounts of data frequently on the Pyra right now is a web browser caching pages and data, and that is not likely to be on the same order unless you’re visiting thousands of new data hungry pages every day. It would be nice to gather some disk usage metrics on the Pyra to set people at ease. Anyone with a prototype interested in doing some tests?

https://source.android.com/devices/tech/perf/flash-wear

In terms of wasting space, the bootloader is < 1MB and the kernel and related files are < 50MB. Right now we keep this on a 256MB partition so there is plenty of room for upgrades and multiple kernels etc, but it’s not necessary to be this large. With the cost of SD cards and such these days, is that really a huge deal? I picked up a 128GB card (which I use for my home partition) for like $30 and the filesystem and partition map overhead alone was greater than that.



Yes. Not much space inside. Those SSDs are large and there’s a lot going on inside.

Fair enough!

 

[ToastBucket]

Fair enough!

I mean you can always take the thing apart yourself when you get one and take a look inside you might be able to hack something up. But no PCIe that I’m aware of is hooked up. Could be wrong though.

 

[God Ginrai]

Like Grench said, it will be a long long time before the eMMC dies. Everyone seems to be very worried about this. Somewhat reasonable, but I think people are overly fearful of this. In my opinion, the performance benefits of the eMMC over the uSD (which are quite significant) outweigh the potential of it dying in maybe 10 years.

SDs are supposed to last users a long time as well, but everyone has had one die on them at one point. AFAIK, these are both using NAND flash, so the risk is just as likely here. The difference is that if we were defaulting to a microSD instead of the soldered memory, we could just replace the microSD if such an event were to happen.

Not to mention if it dies, the device is still usable in 2 other manners (uSD and left SD)

Yes, which is an option between:

1. Having to always hold L1 on boot, or...
2. Your 2 slot device now being effectively a 1 slot device

Both are subpar.

In terms of wasting space, the bootloader is < 1MB and the kernel and related files are < 50MB. Right now we keep this on a 256MB partition so there is plenty of room for upgrades and multiple kernels etc, but it’s not necessary to be this large. With the cost of SD cards and such these days, is that really a huge deal? I picked up a 128GB card (which I use for my home partition) for like $30 and the filesystem and partition map overhead alone was greater than that.

The space isn't the issue here. It's an issue of where these files must live.

-God Ginrai

 

[levi]

SDs are supposed to last users a long time as well, but everyone has had one die on them at one point. AFAIK, these are both using NAND flash, so the risk is just as likely here. The difference is that if we were defaulting to a microSD instead of the soldered memory, we could just replace the microSD if such an event were to happen.

The only SD cards I've even had fail on me is when I formatted them wrongly. Other SD cards I've used I have only stopped using because these days they're too small to be concerned with. I do pay attention to where I buy my SD cards, and where those shops get their stock from. I generally only buy Sandisc and Samsung cards because they're known to have their own controller as well as their own flash fabs, so the odds are they're not getting those parts from dodgy third party sellers and installing them in an SD card case. Other manufacturers like Kingston and Toshiba and others I can't remember can't benefit from their own supplies of parts, so they need to buy in parts and these can and have been effectively faked in the past. I suspect many of the cards that have failed might have been subject to those kind of faked parts.

The space isn't the issue here. It's an issue of where these files must live.

It's a two card device mainly because users are expected to put their dbps on a card in the left slot, which still leaves a slot free for getting photos from your camera or any other use you might have for a spare SD slot. Since the bootloader is so small you could potentially put it on a small partition on the card in the left slot without losing too much space for your dbps and other media.