Lowpowerlab Dual Optiboot

This message was edited 1 time. Last update was at 04/07/2014 03:13:27

Hello,

I'm trying to burn excellent bootloader from Felix Rusu (lowpowerlab) on anarduino:
https://github.com/LowPowerLab/DualOptiboot

In order to archieve wireless programming capability with Felix libraries.
At the moment the flash appears to program correctly but at reboot the atmel chip doesn't reprogram itself.
I know that Flash CS it's on pin D5 on anarduino (vs. pin D8 moteino) and I've changed this to the optiboot.c file

See interesting lines (307-312):
#define FLASHSS_DDR DDRB //
#define FLASHSS_PORT PORTB // must change to PORTD anarduino
#define FLASHSS PINB0 // must change to D5 (PIND5) anarduino
#define SS PINB2 //
#define FLASH_SELECT { FLASHSS_PORT &= ~(_BV(FLASHSS)); }
#define FLASH_UNSELECT { FLASHSS_PORT |= _BV(FLASHSS); }

Anyone is trying this?

Greetings, and thank you for your post.

There is a brand new optiboot rewrite now in the works, and it is nearing the final testing phase.
It will also include new tools for doing wireless distributed remote reprogramming. As soon as this is available, I will post both here in the Forum AND on the miniwireless website portal page.

Best Regards.
Rick

Please skim this post
http://forum.anarduino.com/posts/list/0/13.page#23

steve

This too....

I made a demo video... Introduced as a quick demo. Well, the video is 15 minutes long.
A version is on vimeo.com at this link:

https://vimeo.com/103387066?utm_source=email&utm_medium=clip-transcode_complete-finished-20120100&utm_campaign=7701&email_id=Y2xpcF90cmFuc2NvZGVkfGJiZDUxMjNjNmJhZjUyYjQzNjQ4MWVmMzY3MGNhM2VjMjA2fDI2ODYzODM1fDE0MDc5NzgyMTZ8NzcwMQ%3D%3D

Suggest use of full screen mode to see detail and print.

Steve

Hello Steve,
great video and very impressive demo of your software.

I'm looking forward to when you release it. Do you also support transfer to nodes that only wake up periodically? I ask because all my nodes (except gateway) will go into sleep mode after transmitting data.

thanks for sharing

This message was edited 4 times. Last update was at 15/08/2014 12:53:56

sensorhub wrote:Hello Steve,
great video and very impressive demo of your software.

I'm looking forward to when you release it. Do you also support transfer to nodes that only wake up periodically? I ask because all my nodes (except gateway) will go into sleep mode after transmitting data.

thanks for sharing

The node would have to send a mesage every nth wakeup to the host computer to ask if there are any pending activities, one of which might be "new download (program code or other data) is available".
At the moment, automating all this with no operator at the computer isn't in work because that's very application specific. But the messaging protocols are there and some automation on the computer could be implemented.

Or, in a past project, I used a method where the nodes don't transmit a poll as above. They're passive and this is needed for some wireless regulatory regions of the world - especially for mobile/vehicular nodes. I had the gateway broadcast to all in range nodes a "message is waiting", which is a list of node IDs for whom a message is being held. The nodes wakeup long enough to hear the broadcast and if their ID is in the list, they transmit a message to start a transfer. The usual challenge is how to synchronize the node's wakeup so it will "catch" the message waiting broadcast. I did solve that need on a prior project but it is quite dependent on the battery life conservation strategy used in varying applications. I did put in each broadcast a code to say how long until the next broadcast to help nodes synchronize their sleep times. This sort of mechanism is used in time synchronized mesh and star networks, though with more precision than we need here since a receiver turn-on for x amount of time but quite infrequently (say, n broadcast interval times), is simpler.

Another thing I've thought of is for the gateway to receive from the computer various firmware updates or other bulk data, and store it in the 16MB flash board on the Anarduino MW. Tag the data as to which nodes should get the data. Then the computer isn't involved. The nodes and the gateway could do the transfer later in time.

And vice-versa, for data collection done by a node needing to go to the computer, either directly or by store-and-forward at the gateway.

The gateway could have an ethernet interface so any credentialed computer on the LAN or Internet could access the gateway and thus the network. And/or the "comptuer" could be a SBC like the Raspberry Pi or Beaglebone Black.

Lots of use cases.

This is where I am now (I admit, I'm not so fast as I'm pretty new to this kind of code and it takes pretty much time testing it over and over. Also I don't have access to an logic analyzer). I also have a hardtime reading the flash. Funny enough the flash working perfectly with the arduin spi flash library.

--- Optiboot.c.orig     2014-09-03 06:19:08.000000000 +0200
+++ Optiboot.c  2014-12-05 21:01:54.000000000 +0100
@@ -476,6 +476,8 @@
   #define FLASHSS_PORT    PORTB
   #define FLASHSS         PINB0
   #define SS              PINB2
+  #define MOSI            PINB3
+  #define SCK             PINB5
 #elif defined (__AVR_ATmega1284P__) || defined (__AVR_ATmega644P__)
   #define FLASHSS_DDR     DDRC
   #define FLASHSS_PORT    PORTC
@@ -541,7 +543,9 @@

//SPI INIT
 #if defined(__AVR_ATmega168__) || defined(__AVR_ATmega328P__) || defined(__AVR_ATmega88) || defined(__AVR_ATmega8__) || defined(__AVR_ATmega88__)
-  DDRB |= _BV(FLASHSS) | _BV(SS) | _BV(PB3) | _BV(PB5); //OUTPUTS for FLASH_SS and SS, MOSI, SCK
+  //DDRB |= _BV(FLASHSS) | _BV(SS) | _BV(MOSI) | _BV(SCK); //OUTPUTS for FLASH_SS and SS, MOSI, SCK
+  DDRD |= _BV(FLASHSS);
+  DDRB |= _BV(SS) | _BV(MOSI) | _BV(SCK); //OUTPUTS for FLASH_SS and SS, MOSI, SCK
   FLASH_UNSELECT; //unselect FLASH chip
   PORTB |= _BV(SS); //set SS HIGH
 #elif defined (__AVR_ATmega1284P__) || defined (__AVR_ATmega644P__)

Yes, it's a struggle.

I have the a bootloader working on the mega328 and mega1284 with the Spansion SPI Flash chip. And other code I wrote writes to the flash the downloaded new firmware that was sent via RadioHead's stack on a gateway that connects via a RadioHead stack to a PC's serial port. Then that serial port link (datagram) goes on into a Python-coded version of RadioHead's stack, then into the applications which are GUI based Python. These apps manage all the radios and their configs and distribution of new firmware when needed. The end nodes send unsolicited status messages now and then and that includes what firmware version is in use - so the apps can know which ones need updates.

This message was edited 2 times. Last update was at 06/12/2014 05:50:32

stevech wrote:Yes, it's a struggle.

Yes It definitely is a struggle. Can you give me a hint if I'm on the right track with the SPI initialization and ports/pins?

I have the a bootloader working on the mega328 and mega1284 with the Spansion SPI Flash chip. And other code I wrote writes to the flash the downloaded new firmware that was sent via RadioHead's stack on a gateway that connects via a RadioHead stack to a PC's serial port. Then that serial port link (datagram) goes on into a Python-coded version of RadioHead's stack, then into the applications which are GUI based Python. These apps manage all the radios and their configs and distribution of new firmware when needed. The end nodes send unsolicited status messages now and then and that includes what firmware version is in use - so the apps can know which ones need updates.

This all sounds great, you're making me jealous! How much would you need for a "license" to your software, I'm willing to pay but I will need it soon and it's not for commercial use! My projects have been suspended for too long.

license for non-commercial use would have to be on a case-by-case basis, after review.

Apologies, but there is just too much theft going on this this world we find ourselves in.

Hi Steve,

stevech wrote:
license for non-commercial use would have to be on a case-by-case basis, after review.

Apologies, but there is just too much theft going on this this world we find ourselves in.

But are you willing to put a price on it for personal use? I would also pay for that since it would save me time when implementing my house nodes.

Thanks
JrBenito

This message was edited 1 time. Last update was at 09/12/2014 12:34:05

jrbenito wrote:
But are you willing to put a price on it for personal use? I would also pay for that since it would save me time when implementing my house nodes.

That's what I meant with my question

Hi,

I've published my port of DualOptiboot onto OptiBoot v6.

The code is accessible at https://github.com/abouillot/DualOptiboot6.

Other changes in the support libraries are the Flash Erasing, to be made by 4K pages, as Spanion chip doesn't support 32K block erase.

Be sure to adjust the low fuse to DE, instead of default FF for most arduino boards, to correctly flash after reboot.

The code still needs optimisation, especially on size side

This message was edited 1 time. Last update was at 10/12/2014 09:12:48

mirmit wrote:

I've published my port of DualOptiboot onto OptiBoot v6.

Hi Mirmit,

Thanks for sharing, I'll look at this tonight!

The code still needs optimisation, especially on size side

uncomment the DEBUG define, this should make it fit just barely.

Regards,
William

This message was edited 3 times. Last update was at 13/12/2014 05:38:55

Ok, it seems this code is slightly bigger indeed and just disabling debug is not enough to stay <1024 bytes.

I managed to strip it down a bit to 1034 bytes and it looks like it still works. So a few more modifications are required to get it below 1024 bytes.

--- optiboot.c.orig     2014-12-12 15:29:42.583068166 +0100
+++ optiboot.c  2014-12-13 12:37:10.000000000 +0100
@@ -221,6 +221,8 @@
 /*             verifySpace to use WDT instead of appstart */
 /* 4.1 WestfW: put version number in binary.              */
 /**********************************************************/
+#define LED_START_FLASHES 0
+#define ANARDUINO 1

#define OPTIBOOT_MAJVER 6
 #define OPTIBOOT_MINVER 2
@@ -433,7 +435,7 @@

#define SPIFLASH_BYTEPAGEPROGRAM  0x02        // write (1 to 256bytes)

-#define DEBUG_ON                            // uncomment to enable Serial debugging
+//#define DEBUG_ON                            // uncomment to enable Serial debugging
                                               // (will output different characters depending on which path the bootloader takes)

uint8_t SPI_transfer(uint8_t _data) {
@@ -514,13 +516,17 @@
   SPCR |= _BV(MSTR) | _BV(SPE); //enable SPI and set SPI to MASTER mode

//read first byte of JEDECID, if chip is present it should return a non-0 and non-FF value
+  #ifndef ANARDUINO //JEDEC id doesn't work on anarduino so skip it for now
   FLASH_SELECT;
   SPI_transfer(SPIFLASH_JEDECID);
   uint8_t deviceId = SPI_transfer(0);
   FLASH_UNSELECT;
+  #endif

// Disabled check, as it only returns 0xFF or 0x00 for some reasons
-//  if (deviceId2==0 || deviceId2==0xFF) return;
+  #ifndef ANARDUINO //JEDEC id doesn't work on anarduino so skip it for now
+  if (deviceId2==0 || deviceId2==0xFF) return;
+  #endif

//global unprotect
   FLASH_command(SPIFLASH_STATUSWRITE, 1);
@@ -528,7 +534,8 @@
   FLASH_UNSELECT;

//check if any flash image exists on external FLASH chip
-  if (FLASH_readByte(0)=='F' && FLASH_readByte(1)=='L' && FLASH_readByte(2)=='X' && FLASH_readByte(6)==':' && FLASH_readByte(9)==':')
+  //if (FLASH_readByte(0)=='F' && FLASH_readByte(1)=='L' && FLASH_readByte(2)=='X' && FLASH_readByte(6)==':' && FLASH_readByte(9)==':')
+  if (FLASH_readByte(0)=='F' && FLASH_readByte(2)=='X' && FLASH_readByte(9)==':') //Save bootloader space
   {
 #ifdef DEBUG_ON
     putch('L');
@@ -539,7 +546,7 @@

uint16_t b, i, nextAddress=0;

-    LED_PIN |= _BV(LED);
+    //LED_PIN |= _BV(LED); //Save bootloader space
     for (i=0; i<imagesize; i+=2)
     {
 #ifdef DEBUG_ON
@@ -562,7 +569,7 @@
         nextAddress += SPM_PAGESIZE;
       }
     }
-    LED_PIN &= ~_BV(LED);
+    //LED_PIN &= ~_BV(LED); //Save bootloader space

#if defined(RWWSRE)
     // Reenable read access to flash
@@ -575,8 +582,8 @@

#ifdef ANARDUINO
        // Anarduino doesn't support 32K block erase. Do it, with 8 pages of 4k each
-       int page;
-       long address;
+       uint16_t page;    // modded to unsigned int to save bootloader space
+       uint16_t address; // modded to unsigned int to save bootloader space
        for (page = 0, address=0; page< 8; page++) {
          FLASH_command(SPIFLASH_BLOCKERASE_4K, 1);
          SPI_transfer(address >> 16);

Did you remove the blink-the-LED code? It's 40 or so bytes.

I've gotten my bootloader with support for flashing from UART as well as recall from SPI flash - at about 950 bytes of C code.

This message was edited 1 time. Last update was at 13/12/2014 16:25:29

stevech wrote:Did you remove the blink-the-LED code? It's 40 or so bytes.

I've gotten my bootloader with support for flashing from UART as well as recall from SPI flash - at about 950 bytes of C code.

Do you use Optiboot 5 or 6 as base or a self written bootloader from scratch? The code Mirmit posted is optiboot 6. I allready disabled the led functions and they are ifdef'ed so it's skipped at compile time.

This message was edited 1 time. Last update was at 14/12/2014 00:32:31

I looked at several bootloaders that handle RAMP for AVRs up to 128KB of flash (mega1284p and mega32p). Not interested in mega2560.
I used the write-flash macros (I think they came from Atmel, not sure), and some of the notions for STK500 subset commands.
I wrote the SPI flash I/O from scratch to keep it small.
And the technique for detecting what's in flash and where is my own scheme.

Optiboot has evolved and been tweaked by so many individuals that I chose not to just modify it.

I didn't elect to download directly from the wireless radio link and write to flash... though I did that on a prior bootloader. It was hard to make sure there could be no deadlock with only a partial download, a power failure mid-download, and all sorts of errors. This did work, and it had to... as there are 100+ of these all around the globe, remotely reprogrammed via cellular and IP. The flash size on that was 120KB and there was no mass storage like big RAM or SPI flash. So each time the MCU reset, the bootloader ran again to check if the last remote reprogram was successful (CRC OK, etc.).

This could be done for the little AVR with a 2KB bootloader, I think. Might be best done with a mega1284 as the larger bootloader would have less impact on the 128KB of flash, and the 16KB of RAM will help in buffering for larger sized flash pages. Alas, it's a TODO item. Meanwhile the intermediate buffer is the SPI flash chip which serves other purposes... the bootloader use of the SPI flash is a small percentage of the flash chip's capacity. I'm using the top-most sector in that 16MB flash.

The bootloader fit within 2K and therefore, except if you need more than 30K for your sketches, this is suitable out of the box. The only point to take care of is about the fuses, to accommodate the bootloader size and the boot sequence.

Do the miniwireless boards have the Optiboot bootloader in them stock?

Greg

Yes, that has been my experience.

Thanks, I think I am going to try wireless programing.
You seem to know a lot about the boards. What I did is to order a board with the RFM12B instead of the HW like the others I have. Can I put the CW in place of the 12B that I got by mistake?

Thanks Greg

I vaguely recall that the two do not interoperate.

This message was edited 1 time. Last update was at 25/03/2015 16:54:06

mirmit wrote:The bootloader fit within 2K and therefore, except if you need more than 30K for your sketches, this is suitable out of the box. The only point to take care of is about the fuses, to accommodate the bootloader size and the boot sequence.

How did you make it under 2k? I'm trying to compile code from you repo and the smallest hex I could get is 2870 bytes.

This message was edited 1 time. Last update was at 25/03/2015 19:35:34

I got it into 1KB with a 50 or so to spare. Boot from UART or SPI Flash.
I did so by coding a lot from scratch and omitting what I considered to be historical clutter.

Don't forget that a .hex file is about 2.2 times the size of the actual binary image, since a hex file is binary coded to ASCII hex per an Intel standard.