Brainstorming: Stabilizing (parts of) esp-storage / esp-bootloader-esp-idf

[bjoernQ]

This is just to off-load ideas from my brain and brainstorming - ideally it will get us to a plan towards https://github.com/esp-rs/esp-hal/milestone/15

• other than most/all of our other crates, we rely on eco-system traits a lot here (embedded-storage)- turn them into inherent methods as the stable API, make the embedded-storage impls opt-in (done in Prefer inherent functions for esp-storage / esp-bootloader-esp-idf  #5739)

• support flash encryption (initial support added in Add encrypted write read #5810)

• support secure boot: just for completeness here - should only be a matter of tooling (i.e. espflash)? (see Implement support for Secure Boot V2 espflash#191 , WiFi intialization fails with invalid argument error when secure boot v2 is enabled #4770 (comment) )

• investigate encryption vs MultiwriteNorFlash ( Add encrypted write read #5810 (comment) )

• docs

[bugadani]

Another partially relevant thing: since #4173 it is not possible to read the flash before esp_hal::init. This means there is currently no way to configure CPU clocks based on user preference without a reboot. While this may be solved by improving the clock API, I wonder if this limitation will have other consequences

[MabezDev]

This means there is currently no way to configure CPU clocks based on user preference without a reboot

I'm not sure this is even possible in idf, I can't find any APIs to set any frequencies to dividers, only query current values. The way they seem to configure this is with menuconfig settings 🤔.

[bugadani]

I don't mind writing better software than IDF :) The idea came up that I might want to underclock my EKG to conserve power, but right now I don't have a good way to do it.

[MabezDev]

This is a dump of my internal notes I've been making:

Driver or crate?

As an extension of SPI, this feels like this should be something that should be part of the HAL. However, the impl will likely just call out to rom functions (or will at some point) which puts the driver in a bit of weird catagory.

imo it could be:

• a separate crate esp-storage (imo the name could do with some bikeshedding)
• a driver under spi, esp_hal::spi::master::flash or esp_hal::spi::flash
• a separate crate that uses esp-hal's SPI driver (this gets a bit funky when you consider spi0/spi1 are not GP spi)

APIs

Required:

• init
• read
• write
• erase chip (?)

Maybe:

• erase region
• mmap (map physical flash to bus address) (needed but unlikely to be stable at 1.0)

Fixes

• Soundess fixes (where applicable)
• RAM placement of fn's and types
• concrete API impls, not just embedded storage impls
• LL: mark as unstable

Questions

• Should it work with external flash chips?
• Should it be SPI based, as in to "drive" it, it takes a SPI instance
  • This might help with the first point
• How to expose flash chip specific configs? I.e not all chips will work out of the box
• Does this need integration with OS primitives? I.e how do we signal to multicore chips to stop pulling from the cache? Do we just leave this to the user?

[renkenono]

if I may weigh in as a mere user, I'd like to throw some ideas I have in mind that might add some value to this brainstorming issue. This is obviously biased as it aligns to what I currently desire to have in the future.

Flash encryption

I am very interested in the flash encryption feature. However, I'll likely implement a transitional firmware which would migrate devices already using esp-hal from their unencrypted state to having flash encryption support. This will only be possible if all the necessary eFusers API are implemented, I haven't checked the current state of esp-hal but the end goal is clear.

Secure boot v2

I'm currently experimenting with secure boot v2. However, it's independent from esp-hal which means that for the most part, these suggestions are specific to espflash.

1. I saw initial progress made to manipulate efusers in espflash, that's great! I'd like to be able to replace espefuse burn-key-digest from esptool.
2. espflash should keep automation in mind, because it'll likely be used in hardware provisioning, I mainly use --port $PORT --do-not-confirm when relying on espefuse.
3. espflash flash is pretty cool. However, it's unusable without adding support for --binary instead of unconditionally requiring the ELF. This is because it currently doesn't implement firmware signing and a good first step would be to allow the user to write any arbitrary binary blob while also verifying the constraints imposed by the partition table.
4. Proper support for signing and effectively a replacement of espsecure. Not sure if this should be part of espflash or not.

Again, I haven't checked whether it's currently possible to write an intermediate firmware based on esp-hal which would enable secure boot V2 for the device it is running on, that'd be pretty cool to have (assuming it is feasible to begin with).