MySensors (6)


MySensors smartSleep

How to support the MySensors smartSleep function?

FYI my development board looks like this. Parts include Arduino Pro Mini, MCP1703 (+v.e to 3.3v), 8 channel relay board, nRF24L01+ radio module.

I want to be able to reliably send commands to this MySensors node but I want the node to be sleeping most of the time when the relays are not activated.

When you code up a MySensors node you can save power by sleeping. This works fine until you want to send the node a command. The node won’t receive the command because it is asleep. I initially thought that I could get around this by simply sending the command with a qos status of 1.  This doesn’t work!

Recall that a MySensors NODE communicates with a MySensors GATEWAY. The gateway is responsible for sending and receiving messages to an MQTT BROKER.

If you send a command with a qos status of 1 this guarantees delivery to the GATEWAY but it does not guarantee delivery to the node if the node is sleeping. The gateway isn’t clever enough to buffer messages for nodes.

The MySensors library has a built in function called smartSleep. When you call this function the following happens:

  1. The Node sends a message (I_PRE_SLEEP_NOTIFICATION) to the server with a payload containing the number of milliseconds before it sleeps
  2. The Node waits for this amount of time
  3. The Node sleeps

Sending a command to a sleeping node then becomes a lot easier.

The following algorithm can be used when a command is received.

  • MqttCogs receives a set command
  • Check whether node is sleeping
  • If sleeping buffer message to new table
  • If not sleeping or it doesn’t look like a MySensor packet then issue the set command

The following algorithm can be used when a incoming I_PRE_SLEEP_NOTIFICATION message is received by MqttCogs

  • Check buffer for messages for this node (extract in datetime order)
  • Send messages to Node

I’ve had an alpha stab at implementing this. The following will submit a set command to node 10, sensor 6. This is a relay, the value is the number of minutes the relay should open.

[ mqttcogs_set topic='mysensors_in/10/6/1/0/2' text='Add' label_text='Enter a new value here:' input_type='number' input_min='0' input_max='6' input_title='Please enter a number between 0 and 6' qos='1'/ ]

And heres a history of what happened. When you set the value the graph won’t change. The node is only not sleeping for 500ms every minute unless the relay is activated….

For reference, my arduino loop looks something like the code below. I added in a couple of useful safety features.

  • A relay can only be activated for a fixed time only.
  • When the arduino restarts the relays are set to OFF
void loop()
 {
 
 if (sensor_active) {
     //check if sensor should still be on
     if ((millis() - sensor_start)> sensor_ms) {
        digitalWrite(sensor_active-1 + RELAY_PIN, RELAY_OFF);
        saveState(sensor_active, false);
        send(msg.setSensor(sensor_active).set(false));
        sensor_active = 0;
        sensor_start = 0;
        sensor_ms = 0;
     }
     //don't sleep while relay is on
     return;
   }
   smartSleep(60000);
 }

So, power saving?

It’s good. While sleeping the board (without modifications) consumes 1.5mA (mainly due to the led). I removed the on LED and the board now consumes 60uA when sleeping (should be less so need to look at why).

When relay is on it uses *a lot* because the relay has to be energised.

I’ve coded things so that only one relay can be on at once. This reduces the suck through the MCP1703. The MCP1703 only supports 250mA current draw. More than 2 relays on at once would break it!




Connect Serial Flash to Arduino

How to Connect Serial Flash to Arduino to Save Power

I chose a SPI Flash or Serial Flash module for 2 reasons. Firstly, it uses a lot less power than an SD card and secondly it has a much smaller form factor. An SPI Flash module will use <10mA for writing whereas an SD card will use up to 150mA.

So my thought was ‘I can drive the Serial Flash module from one of the Arduino digital IO pins’.

I bought one of these https://www.ebay.co.uk/itm/W25Q-Winbond-Serial-Flash-Memory-Module-SPI-W25Q128B-BIOS-25Q64BVSIG-EEPROM-/181873964697. This contains a winbond 128Mbit Serial Flash chip.

The board includes pull up resistors (more about these later) and labelled pins. The board is 3.3v volts. I did all my testing for this post using a Arduino Mini Pro at 3.3v so I didn’t have to do any logic level conversion.

Why 128Mbits?

One of my previous posts included code to write a serial buffer to EEPROM. This allowed chunks of data up to 255bytes to be written sequentially to the EEPROM. On startup the code determined the next ‘free page’ to write to and only overwrote one 4K block once the log started to loop.

So 128Mbits = 16777216 bytes

So 16777216 bytes = 65536 lots of 255byte records

So if I write a single log record every minute that is 45 days of logging

Wiring the Winbond W25Q128B module to the Arduino

Wiring the SPI Flash Module
Arduino Pin SPIFlash Pin
Pin 6 (your choice) VCC
GND GND
VCC WP/IO2
VCC HOLD/IO3
MISO DO/IO1
MOSI DI/IO0
SCK CLK
Pin 10 (SS) CS

Enable and Disable

I firstly tried to the SPIMemory library for sleeping. There are functions for powerUp() and powerDown(). I rejected these. A powerDown and Arduino reset causes the SPI Flash module to lock up and the Arduino wouldn’t restart.

This took a bit of trial and error but I eventually came up with the following function. I’ve attached VCC of the Serial Flash to Pin 6 of

void flashToggle(bool yes) {
  if (yes) {
      pinMode(6, OUTPUT); 
      digitalWrite(6, HIGH); 
      flash.begin();
  }
  else {
      pinMode(6, INPUT); 
     digitalWrite(6, LOW); 
  }
}

Switching the pin to INPUT and setting low seemed to stop all power from the pin. I tried all other options.

Power  Use

These were measured on the VCC line with a cheap multimeter.

Continuous write ~ 5mA
after flashToggle(false) – 0uA (err, yes really. Nothing)

Conclusion

For me, a result. The module uses a tiny about of power and I can switch it off by simply toggling the DIO pin. I need to see if the same is true when it is attached to a 5v Arduino through a logic level converted.




Arduino Circular Buffer to Serial Flash

Writing an Arduino Circular Buffer to Serial Flash.

Data is arranged in BLOCKS of 4K, each block contains PAGES of length 256 bytes. Pages are written sequentially to flash.

When we get to the end of the flash we then start at the beginning. Serial Flash must be erased a BLOCK at a time. We erase only one BLOCK ahead of where we are currently writing.

On startup we determine what is the next free page in the flash so that we can continue writing. We keep an address pointer in memory after this.

The first byte of each page is used to flag teh following:

11111111 – not written

01111111 – writing

00111111 – written

The following 255 bytes contain the page payload. We CANNOT write more than 255 bytes to a page.

#define BLOCK_SIZE 4096  //4K
#define PAGE_SIZE 256    //

#define PAGE_WRITING 0x7F //01111111
#define PAGE_WRITTEN 0x3F //00111111

SPIFlash flash;
uint32_t addr;
uint32_t capacity;

void setup() {
  Serial.begin(BAUD_RATE);
   flash.begin();
   capacity = flash.getCapacity(); 
   addr = getNextAddress();
}


/**
  Writes a variable length packet of no more than 255
  bytes to the rolling log

  @param data_buffer[] the bytes to write
  @param bufferSize the size of data_buffer
  @return the next available address
*/
uint32_t writePacket(byte data_buffer[], int bufferSize) {
  if (addr%BLOCK_SIZE==0) {
     Serial.print("ERASE:");
     Serial.println(addr);
    flash.eraseSector( (addr + BLOCK_SIZE)%capacity);
  }
  flash.writeByte(addr, 0x7F); //flag as started
  flash.writeByteArray(addr+1, data_buffer, bufferSize);
  flash.writeByte(addr, 0x3F); //flag as written
  return (addr + PAGE_SIZE) % capacity;  
}


/**
  Examines the SerialFlash finds the next empty page. Should
  be called on startup. A reference can then be kept to
  log position.

  @return the next available address
*/
uint32_t getNextAddress() {
    uint32_t ad = search(0,capacity, BLOCK_SIZE, 0x3F);
    return (search(ad, BLOCK_SIZE, PAGE_SIZE, 0x3F)+ PAGE_SIZE)%capacity;
}

/**
  Examines the first byte in a chunk of flash to determine if 
  page has been written.

  @param from the from address
  @param to the to address
  @param ss the step size, normally BLOCK or PAGE
  @param headermask mask to apply to first byte of PAGE
  @return the next available address
*/
uint32_t search(uint32_t from,uint32_t to, uint32_t  ss, byte headermask) {
  byte lb = 255;//isnt written
  uint32_t pt;
  
  for ( pt = from; pt<=to;pt=pt+ss) {
    byte b = flash.readByte(pt);   
    if ((b==255) && (~lb & ~headermask)) {
      return pt-ss;
    }
    lb = b;
  }
  //got to the end
  return pt;
}



MySensors Solar Powered Node – Part 3

There’s been a bit of gap since my last post. Two items have been particularly problematic. Firstly, accurately determining the battery voltage and secondly getting around the low power startup failure.

I got a bit sidetracked. I tried to use a Thyrsitor to switch current on/off to the MCP1703. This was triggered by the voltage from the Supercapacitor. This doesn’t work!

A Supercapacitor doesn’t behave like a battery. Once current is drawn, there is an immediate voltage drop. My Thyristor testing showed that this simply caused a on/off/on/off…. sequence to occur. As soon as the thyristor switched on the voltage dropped, the gate on the thyristor closed and power was cut. At low light levels the power from the solar panel just isn’t enough to stop the voltage from dropping.

I managed to get around the Thyristor problem by using the Arduino to determine when to draw power. More on this later on.

Getting the Maximum Solar Voltage

I want the supercapacitor to charge as quickly as possible. In order to do this the open supply voltage from the solar panel needs to be as close to 5.5v as possible. It occurred to me that the diode that I first chose was not very clever. It has a 0.7v voltage drop for most of the time. A 0.7v drop at low light levels is significant. It can be the difference between the circuit switching on or off.

I replaced the existing diode with a STMicroelectronics BAT43, Schottky Diode, 30V 200mA. This has a peak forward voltage drop of 450mV.

This will be obvious to a lot of fokes. Not to me!

Protecting the Supercapacitor

As the voltage drop with the above diode is now much smaller it is possible the solar voltage could rise above 5.5v. I used a zener diode to clamp the voltage across the Supercapacitor to 5.6v. This should be 5.5v but I couldn’t source a component. Zener diodes do nothing until the voltage rises above their rated value and then they start conducting (backwards…).

Low Voltage Start

This was a big, big problem. I’ve explained how my thyristor solution didn’t work. I went back to trying without the thyristor. It still didn’t work! As soon as there is enough power for the Arduino to start, the MySensors library switches on the radio and tries to ‘present’ the node. This is power hungry, too much current is drawn and the radio initiation fails. The MySensors library then goes into startup loop until the supercapacitor is dead.

I got around this by modifying my sketch. In the event that the start voltage is too low I power down the radio and then sleep. This works!

Here’s an example of the hackery I added to the MySensors before statement to power down the radio if the voltage isn’t high enough.

void before()
{ 
  analogReference(INTERNAL) ;
  
  for (;;) {  
   int sensorValue = analogRead(BATTERY_SENSE_PIN);
   sensorValue = analogRead(BATTERY_SENSE_PIN);
   float batteryPcnt = static_cast<float>(static_cast<int>(sensorValue*10.0 / 10.23))/10.0;
   int batteryV = static_cast<int> ( batteryPcnt*55);

   if (batteryV> ON_BATTERY_VOLTAGE) 
    break;
           
    transportInit();
    transportPowerDown();
    hwSleep(CHARGE_SLEEP_TIME);
   
    if (preHwInit) {
      preHwInit();
    }
    hwInit();
   }
}

Measuring Voltage using Arduino

Should be easy right? All we have to do is attach a voltage divider to one of the analogue pins. We know that the maximum input voltage should be 5.5v. all  we need to do is select the resistances so that the tax point is 1.1 volts. We’ll obviously use the Arduino internal reference as we can’t rely on the VCC voltage on the Arduino.

I’ve had some odd results. I chose R1 = 1.5M and R2 = 0.377M (47K + 330K). One of my test Arduinos works fine the other systematically reports lower than expected voltages at the tap point. I have no idea why.




MySensors Solar Powered Node – Part 2

I prototyped my Solar Powered Arduino design and started taking some measurements. I’ll show the (really bad) performance of my cheap solar panel, give you an idea of how much current my Arduino and MCP1703 suck and then explain my first stab at a sketch. Remember, this is work in progress and if you read the next post about my Solar Powered Arduino you’ll find out that I came across a few problems that I had to get around.

Solar Powered Arduino – some baselines

Solar Panel

Panel is rated at 6v 150mA ~ 1W. Here are some readings from the panel this morning. It’s a very cloudy midwinter day here. You can see that the panel is <1% efficient at times.

Measured Solar Panel Output
conditions voltage (mV) current (mA)
Very cloudy midwinter 8.30am 4.2 0.510
Very cloudy midwinter 10.00am 5.0 0.810
Very cloudy midwinter 10.30am 5.63 1.9
Very cloudy midwinter 12.30am 5.02 0.8

Arduino

This is a 3.3v Arduino clone with a couple of minor modifications.  The Power LED is removed, the onboard regulator is removed. These are the standard hacks to reduce the board’s power consumption. The bootloader is not modified as we are using the MCP1703 regulator.

It has a standard nRF24L01+ clone attached to it and is running the MySensors 2.1.0 software stack. Here’s an idea of power consumption in different scenarios. These are tested using my cheap multimeter so the uA value is probably not quite right. They are also tested before the MCP1703 as I want to get an idea of total power consumption.

Measured Arduino Power Consumption
Arduino Radio current (mA)
Awake On 30
Arduino Standard Sleep Off 4
MySensors Sleep Off 0.0077

The Solar Panel

It’s worth mentioning this part of the system. I’ve got a few worries here. I ordered a 1N4148TR diode. This is a very cheap low current device (max 300mA), double the rated output of the solar panel. The plan was that the diode was used to stop leakage from the capacitor back through the solar panel when it’s dark. It is also used to drop the voltage from the solar panel to ‘below’ the rated value of the Super Capacitor.

I think I’ve chosen the wrong diode. After measuring the voltage drop with no load, it appears to be just under 0.5v – not enough! As the current increases the voltage drop increases to 0.7v. Ideally I need to find one with a min voltage drop of 0.6v, or work on another solution. This is an interesting post about the 4148 diode characteristics.

I think what the article is telling me is that my diode has a bigger voltage drop at lower voltages and currents. I need either a diode with a fixed minimum voltage drop or a clamping circuit like this to make sure I keep within my voltage limits.

The Sketch

Here’s the example sketch for the Solar Powered Arduino. There are some things to note. If the Capacitor is in a good state then values for battery percentage and voltage are sent to the server every 5 minutes. In the case where the battery voltage has dropped too low, no values are sent and the Arduino sleeps for 15 minutes before checking again.

// Enable debug prints to serial monitor
#define MY_DEBUG 

// Enable and select radio type attached
#define MY_RADIO_NRF24
//#define MY_RF24_PA_LEVEL RF24_PA_LOW

#define MY_NODE_ID 98
//#define MY_PARENT_NODE_ID 0
//#define MY_PARENT_NODE_IS_STATIC

#include <MySensors.h>
#include <SPI.h>

#define OFF_BATTERY_VOLTAGE 3500
#define ON_BATTERY_VOLTAGE 3800

int BATTERY_SENSE_PIN = A0;         // select the input pin for the battery sense point
unsigned long UPDATE_SLEEP_TIME = 300000;  // sleep time between reads (seconds * 1000 milliseconds)
unsigned long CHARGE_SLEEP_TIME = 900000;

MyMessage msg(0,V_VOLTAGE);

void before()
{
  #ifdef MY_DEBUG
   Serial.println("Node STARTING ");
   #endif
}

void presentation() { 
  // Send the sketch version information to the gateway and Controller
  sendSketchInfo("Solar board", "1.1"); 
}

void loop()     
{     
   // get the battery Voltage
   int sensorValue = analogRead(BATTERY_SENSE_PIN);
   
   float batteryPcnt = static_cast<float>(static_cast<int>(sensorValue*10.0 / 10.23))/10.0;
   int batteryV = static_cast<int> ( batteryPcnt*50);

#ifdef MY_DEBUG
    Serial.print("Voltage: ");
    Serial.println(batteryV);
#endif
     
   if (batteryV< OFF_BATTERY_VOLTAGE ) {
#ifdef MY_DEBUG
      Serial.println("Voltage POOR - SLEEPING");
#endif
     sleep(CHARGE_SLEEP_TIME);
     return;
    }

   sendBatteryLevel(batteryPcnt);    
   msg.setType(V_VOLTAGE);
   send(msg.setSensor(0).set(batteryV));

#ifdef MY_DEBUG
   Serial.print("Battery Voltage: ");
   Serial.print(batteryV);
   Serial.println(" V");

   Serial.print("Battery percent: ");
   Serial.print(batteryPcnt);
   Serial.println(" %");
#endif
   sleep(UPDATE_SLEEP_TIME);
}

Problems!

My Solar Powered Arduino with the sketch and circuit above does work – mostly. During light periods the Super Capacitor is charged as you would expect. During low light periods the voltage across the Solar Panel drops and the diode stops the panel acting like a resistor.

Voltage Clamping

As mentioned before the voltage drop across the diode is not enough to protect the Super Capacitor. I think that this problem can be solved by voltage clamping the solar panel using a zener diode. This would also allow panels of higher voltages to be used.

MCP1703 should be MCP1700

If a voltage of less than 6v is used after the diode then a MCP1700 would be better suited. This has better performance at lower voltages and wastes less power.

Cheap Super Capacitor

I bought my Super Capacitor from eBay for about £3.50. I’ve eventually found some information on it’s performance (I think). I’m going to test but I think that it has pretty poor performance. It leaks quite a bit. I’ll have to work out how to test this and detail some more results

Low Voltage Problems

This is a problem. When it’s dark and the Super Capacitor voltage has dropped below the threshold required to run the circuit, the system never recovers. The Super Capacitor sits at a voltage of about 1.6v and never charges.

My assumption is that at this voltage the Arduino starts to continuously suck power (even though it is not starting) over the amount that the solar panel can provide. A scenario of starting….failing….starting….failing…starting occurs and keeps the Super Capacitor from charging.

I need a solution for this. Any thoughts?




MySensors Solar Powered Node – Part 1

For now, this post is going to contain a few notes regarding a solar powered MySensors node that I put together last weekend. This may not work, so the post will contain a basic circuit and readings from the setup over the next couple of weeks.

Basic Idea

Hooking together Solar (PV) Panel + Supercapacitor + MCU1703 + Arduino

The idea is to power a MySensors node (Arduino Mini Pro with NRF24L01+ radio) using a small solar panel. A Supercapacitor will be used in place of a rechargeable battery. The Solar Panel and Supercapacitor must be carefully balanced as voltage into the Supercapitor shouldn’t exceed the rated value.

The circuit will include a really basic resistor divider so that the arduino can measure the input voltage to the regulator. The sketch is setup to send back the voltage on the AO pin every 5 mins (more about this later). Here is a basic rundown of the parts:

Solar (PV) Panel: 6v 1W rating, 8cm x 5cm, v.cheap!

Supercapacitor: 5.5v rating, 4F.

Regulator MCP1703: Converts input voltage to regulated 3.3v output. Uses only 2uA (micro amps) when there is no load!

Resistor Divider: Simple resistor divider using spare 100K resistors. Converts 5.3v to 3.3v. Sensed on A0 pin on Arduino

Arduino Mini Pro 3.3v: Arduino Mini Pro with Connected NRF24L01+ radio. Running 2.1.0 MySensors code. Radio links to my gateway which pushes messages to MQTT broker and then this blog using MqttCogs

It’s midwinter here. Very cloudy, no sun and lots of rain. Ideal time to check out a solar panel powered Ardiuno!

Prototype

….and this is what it looks like. All pushed together on a breadboard.


Here’s a list of questions that I want answered over the next few days/weeks.

  • Solar Panel voltage after diode must be less than rating for Supercapacitor. Need to check voltage drop across diode when no current is drawn
  • Need to check Arduino current draw with radio on, radio off and when sleeping
  • Need to check Solar Panel current and voltage in different conditions
  • Need to record how quickly Super Capacitor discharges when no solar panel attached
  • Need to record charging rate of Super Capacitor in different conditions
  • Need to test behaviour of system when Super Capacitor is unable to drive the Arduino properly

 

Live Super Capacitor Voltage

Here’s a couple of graphs using MqttCogs. The first shows the Super Capacitor voltage in the last 2.4 hours, the second the voltage in the last 24 hours.

[mqttcogs_drawgoogle options="{series: {0:{labelInLegend: 'Capacitor Voltage(mV)'}}, animation:{startup:true,duration:5},curveType:'function',title:'(98) Supercap voltage mV (last 2 hours)',height:200, hAxis: { title: 'DateTime(UTC)',format:'dd MMM HH:mm' }, vAxis: { title: 'Voltage (mv)' }}" charttype="LineChart" ]]
[[mqttcogs_data from="-0.2"  limit="999" topics="mysensors_out/98/0/1/0/38"/]][[/mqttcogs_drawgoogle]

[mqttcogs_drawgoogle options="{series: {0:{labelInLegend: 'Capacitor Voltage(mV)'}}, animation:{startup:true,duration:5},curveType:'function',title:'(98) Supercap voltage mV (last 24 hrs)',height:200, hAxis: { title: 'DateTime(UTC)',format:'dd MMM HH:mm' }, vAxis: { title: 'Voltage (mv)' }}" charttype="LineChart" ]][ [mqttcogs_data from="-1"  limit="999" topics="mysensors_out/98/0/1/0/38"/]][[/mqttcogs_drawgoogle]