Uploading Data With Python

This page describes how to send data to a Manylabs data set using Python. We'll assume the data is coming from an Arduino attached to the same computer running the Python script. If you need help with any steps e-mail us at support@manylabs.org

Setup Arduino

To get data from the Arduino in Python, the Python script will open the serial port the Arduino is attached to and read any text printed by the Arduino. So the first important piece is that your Arduino needs to print the data you'd like to send.

In the setup function of your Arduino sketch include this code:

Serial.begin( 9600 );

This opens the serial port for sending data at 9600 baud. Then, in your loop function you'll need to print the data you want to send using Serial.print(). We normally create a function like this to handle printing serial data:

// send data to computer over USB/serial
// this will send lines that look like: "name: value, name: value\n"
void sendSerial() {
  // print the name for this value
  Serial.print( "Temp: " );
  // because temp is a float, the 2 means "print the value with 2 decimal places"
  Serial.print( temp, 2 );
  // print the next value with a comma before the name
  // all values after the first should have a comma before the name
  Serial.print( ", Humidity: " );
  // print the next value, also with 2 decimal places
  Serial.print( humidity, 2 );
  // print the \n character that finishes the line
  Serial.println();
}

Then call this function in the loop function:

void loop(){
    // update values...
    // other code ...
    sendSerial();
}

Important The data must be sent in the format: "name: value, name: value\n"

You can find more information on Serial.print here

Create a Data Set

Now you need to create a data set. You can do this in the data hub by clicking Add New Data Set - this will open the data set editor. In the editor, give your data set a name and description, and select Remotely Updated. (Remotely Updated lets other tools know that this data set is updated from an outside source.)

Permissions

Next, set the Sharing Mode as you like. Set the Data Entry Mode to anyone can enter data. Soon we'll switch to the new Manylabs Data API which will remove this requirement.

Important You must currently set the Data Entry Mode to anyone can enter data.

Columns

Now you need to add a few columns to the data set. To add a column click Add New Column at the bottom of the data set editor. Your first column must be called timestamp and have a type of timestamp. The other Manylabs tools use this column to order the data.

Then you need to add a column for every value you're printing from your sketch. For instance, if your sketch will print out a temperature and humidity value, you'll need to add two columns. The name of these columns must match the name your sketch prints. The example Arduino code above prints values with the names "Temp" and "Humidity". So here, the columns would be named "Temp" and "Humidity". The type should be set to numeric for both columns since they're numeric values. You can optionally set units and the number of decimal places to show for each column.

When you're done adding columns, click Save under Remotely Updated - you'll return to the Data Hub.

Data Set ID

To send data to your data set, you'll need to know its ID on the server. To find the id, locate your data set in the data hub and click edit under it. You'll be brought back to the data set editor. The URL in the location bar of your browser will look something like this:

https://www.manylabs.org/data/<data set id>/edit/

Make a note of the data set id from the URL.

Setup Python Script

Download a copy of the python script here: ManylabsDataStream.py

Open ManylabsDataStream.py in a text editor and replace DATA_SET_ID with the data set id you noted earlier. Next, replace SERIAL_PORT with the port your Arduino is connected to.

If you're not sure what the port is, you can open the Arduino Programmer and click Refresh Ports (the button next to the Port dropdown). Then connect your Arduino and click Refresh Ports again. The new port will be your Arduino.

You can also adjust UPDATE_INTERVAL_SECONDS to change how often the script sends data to the server.

Finally, run the script in a terminal. The script will print the values it sends to the terminal window. Now you can open your data set in the data hub.

Troubleshooting

Here are a few things to double check:

  • Make sure your Arduino sketch is printing values that are arranged like this: "name:value, name2:value2". If you open the Serial monitor with your sketch running, you should see these lines printed out.
  • Make sure your data set column names are the same as the names your sketch is printing. If your sketch is printing something like "Temp:22.0, Humidity:50", your columns should be named Temp and Humidity
  • Check that you've set the correct data set id in the Python script.
  • Double check the Serial port you've set in the Python script.

If you're still having trouble, please e-mail us at support@manylabs.org

© 2015 Manylabs
Creative Commons License