MeasurementFile

Last modified by Michael Sibayan on 2016/04/25 12:41
As of January 2016 the Measurement File format has been changed from the old tab delimited variables, to the new JSON format.

The "Measurement File" or "m-file" is an ASCII text, human readable, JSON format file which holds information pertaining to a measurement or group of measurements. In general, a measurement file will hold all data from sensors sampled at a particular time (i.e. - measure a group of five thermocouples at 11:30am on Dec. 1, 2012). However, it is not limited to this, as one could group a single sensor, or even a group of sensors, over an extended period of time. The format of this file is a JSON array of JSON objects containing information about the sensor and it's measurement.

Below is an example of a sensor named "MySensor" that measures speed and direction with raw measurements of voltage and counts.

[{"sensorcode":"MySensor","sensorid":102,"variablecode":"Volts","variableid":1,"value":1.54,"units":"V","DateTime":"2016-03-18 19:00:00"}
,{"sensorcode":"MySensor","sensorid":102,"variablecode":"Direction","variableid":3,"value":224,"units":"Degrees","DateTime":"2016-03-18 19:00:00"}
,{"sensorcode":"MySensor","sensorid":102,"variablecode":"Counts","variableid":1,"value":6,"units":"#","DateTime":"2016-03-18 19:00:00"}
,{"sensorcode":"MySensor","sensorid":102,"variablecode":"Velocity","variableid":4,"value":0.112,"units":"m/s","DateTime":"2016-03-18 19:00:00"}
]

Useful fields:

  • SensorCode - Unique name of a sensor (string)
  • VariableCode - Unique name of a variable (string)
  • Value - Numeric value of the measurement (float)
  • Units - Abbreviation of the units of measure (string)
  • DateTime - Time stamp of the measurement in format YYYY-MM-DD HH:MI:SS (string)
  • SensorID - Database specific id for the sensor (unsigned integer)
  • VariableID - Database specific id for the variable (unsigned integer)
  • DBID - The user account that manages the sensor information (string)
  • DataTable - The table within the schema to which data is stored (string)
The information below is provided for reference and is no longer the standard format.

The "Measurement File" or "m-file" is an ASCII text, human readable, tab-delimited file which holds information pertaining to a measurement or group of measurements. In general, a measurement file will hold all data from sensors sampled at a particular time (i.e. - measure a group of five thermocouples at 11:30am on Dec. 1, 2012). However, it is not limited to this, as one could group a single sensor, or even a group of sensors, over an extended period of time.

The measurement file has strict conventions. Measurement files uploaded to the LEO Data Server must follow these guidelines

  • File must be properly named using a descriptor and a timestamp with file extension of .txt in the format DESCRIPTION_TIMESTAMP.txt
    • The description can contain any valid file name characters EXCLUDING an underscore character “_” such as “My-Description_TIMESTAMP.txt” or “ThisIsADescription_TIMESTAMP.txt”
    • The TIMESTAMP must follow an underscore and must follow the format YYYYMMDD. The timestamp can also contain additional timestamp data such as hour, minute, and second information, however, this must trail the initial required section (Description_YYYYMMDDhhmmss.txt)
    • The timestamp is used to organize processed files into a date based hierarchy file structure.
  • All data within the file must be tab delimited ASCII text. The format for a measurement is:
    • “TIMESTAMP <tab> SENSORCODE <tab> datavalue0 <tab> datavalue1 <tab> … datavalueN”
    • The TIMESTAMP must be formatted “YYYY-MM-DD hh:mm:ss”
      • NOTE: the data loader removes the seconds from the timestamp
    • SENSORCODE must be defined in the database and available in the dataloader application before processing. If a SENSORCODE is not recognized the file will fail to process and get moved to the “Error” output folder.
    • Data values must match the predefined variable output order and should contain data for missing values (the software will insert the “missing data value” as defined in the database for any trailing missing values).

The current convention for file naming is “DeviceName_TimeStamp.txt”, and example filename is “cRIO-LEO-Center-P01_20131121173002.txt” for a device named “cRIO-LEO-Center-P01” and a timestamp 2013/11/21 at 17:30:02. An example of the file contents for a measurement file containing data for 8 sensors with 14 data points is shown below:

2013-11-21 17:30:01        LEO-C_10_-4_4_GMM222           0.121902              341.325300
2013-11-21 17:30:01        LEO-C_10_4_4_GMM222             0.142809              399.866131
2013-11-21 17:30:01        LEO-C_6_-4_1_HFP-1                     -0.001208             -18.838927
2013-11-21 17:30:01        LEO-C_6_0_1_HFP-1                      0.001314              20.860207
2013-11-21 17:30:01        LEO-C_2_-3_0_Model3130          0.000751              55597.257283
2013-11-21 17:30:01        LEO-C_0_4_0_PE102                      0.000000              0.000000
2013-11-21 17:30:01        LEO-C_6_-4_1_TCAV                      23.783472
2013-11-21 17:30:01        LEO-C_6_0_1_TCAV                       23.336414

Tags:
Created by Michael Sibayan on 2014/11/14 09:09

This wiki is licensed under a Creative Commons 2.0 license
XWiki Enterprise 6.0.1 - Documentation