endaq.ide Usage Examples#

Note: For brevity, the following examples assume everything has been imported from endaq.ide:

from endaq.ide import *

Opening IDE files: endaq.ide.get_doc()#

endaq.ide includes a convenient shortcut for importing IDE data: get_doc(). It can load data from local files, or read data directly from a URL.

doc = get_doc("tests/test.ide")
doc1 = get_doc("https://info.endaq.com/hubfs/data/surgical-instrument.ide")

IDE files can be retrieved directly from Google Drive using a Drive ‘sharable link’ URL. The file must be set to allow access to “Anyone with the link.”

doc2 = get_doc("https://drive.google.com/file/d/1t3JqbZGhuZbIK9agH24YZIdVE26-NOF5/view?usp=sharing")

Whether opening a local file or a URL, get_doc() can be used to import only a specific interval by way of its start and end parameters:

doc3 = get_doc("tests/test.ide", start="5s", end="10s")

Accessing measurement data in a Dataset/IDE file#

An enDAQ device consists of many different sensors, and enDAQ devices record their measurement data into separate Channels that correspond to the sensor taking the measurement. This is done because each Channel samples at a different rate, so while Channel 59 (the Control Pad Pressure/Temperature/Humidity sensor) samples at 10 Hz, Channel 8 (the main analog accelerometer channel) may sample at 20000 Hz. Channels themselves consist of different subchannels, which may be different axes (X, Y, Z) or completely different measurements like temperature and pressure. All subchannels in a channel are sampled at approximately the same time.

The Channel data are stored in the channels property of a Dataset, which is returned from the to_pandas(doc)(). Visit our internal documentation for some quick tips on Pandas, or go straight to the source.

import endaq.ide
# Read in a doc
doc2 = endaq.ide.get_doc("https://drive.google.com/file/d/1t3JqbZGhuZbIK9agH24YZIdVE26-NOF5/view?usp=sharing")
# List the available Channels
print(f"{doc2.channels=}")
# Convert the Control Pad Pressure/Temperature/Humidity Channel (Channel 59) to a Pandas DataFrame
control_pad_data = endaq.ide.to_pandas(doc2.channels[59])
# Print the subchannel names
print(f"{control_pad_data.columns=}")
# Print the max and min temperatures seen
print(f"Max Temp={control_pad_data['Control Pad Temperature'].max()}, Min Temp={control_pad_data['Control Pad Temperature'].min()}")

The output of the above code is:

doc2.channels={32: <Channel 32 '16g DC Acceleration': Acceleration (g)>, 80: <Channel 80 '8g DC Acceleration': Acceleration (g)>, 36: <Channel 36 'Pressure/Temperature': Pressure (Pa), Temperature (°C)>, 70: <Channel 70 'Relative Orientation': Quaternion (q)>, 59: <Channel 59 'Control Pad Pressure/Temperature/Humidity': Pressure (Pa), Temperature (°C)>, 76: <Channel 76 'Light Sensor': Light (Ill), Light (Index)>}
control_pad_data.columns=Index(['Control Pad Pressure', 'Control Pad Temperature'], dtype='object')
Max Temp=24.899999618530273, Min Temp=24.260000228881836

Note that by default, to_pandas(doc)() uses datetime for the index format, meaning the measurements are accessed based on the absolute time they were recorded. Users often prefer to access the data using timedelta, the amount of time since the recording started. Using this, to get the duration of the Control Pad data and the average of the first 5 seconds, we could use:

import endaq.ide
import pandas as pd
# Read in a doc
doc2 = endaq.ide.get_doc("https://drive.google.com/file/d/1t3JqbZGhuZbIK9agH24YZIdVE26-NOF5/view?usp=sharing")
# Convert the Control Pad Pressure/Temperature/Humidity Channel (Channel 59) to a Pandas DataFrame
control_pad_data = endaq.ide.to_pandas(doc2.channels[59], time_mode='timedelta')
# Print the time duration
print(f"Duration={control_pad_data.index[-1]-control_pad_data.index[0]}")
# Print the mean of the first 5 seconds
print(f"{control_pad_data[pd.Timedelta(seconds=0):pd.Timedelta(seconds=5)].mean()}")

The output of the above code is:

Duration=0 days 00:00:17.931518
Control Pad Pressure       101728.414991
Control Pad Temperature        24.607073
dtype: float64

Summarizing IDE files: endaq.ide.get_channel_table()#

Once an IDE file has been loaded, get_channel_table() will retrieve basic summary information about its contents.

Some environments, such as Jupyter Notebook or Colab, will automatically render and display the channel table data. From inside the standard Python interactive interpreter, use get_channel_table(doc).data to display the raw information, or to access the table’s contents directly as a Pandas DataFrame.

get_channel_table(doc)
  channel name type units start end duration samples rate
0 32.0 X (16g) Acceleration g 00:00.0952 00:19.0012 00:18.0059 7113 393.86 Hz
1 32.1 Y (16g) Acceleration g 00:00.0952 00:19.0012 00:18.0059 7113 393.86 Hz
2 32.2 Z (16g) Acceleration g 00:00.0952 00:19.0012 00:18.0059 7113 393.86 Hz
3 80.0 X (8g) Acceleration g 00:00.0948 00:19.0013 00:18.0064 9070 502.09 Hz
4 80.1 Y (8g) Acceleration g 00:00.0948 00:19.0013 00:18.0064 9070 502.09 Hz
5 80.2 Z (8g) Acceleration g 00:00.0948 00:19.0013 00:18.0064 9070 502.09 Hz
6 36.0 Pressure/Temperature:00 Pressure Pa 00:00.0945 00:19.0175 00:18.0230 20 1.10 Hz
7 36.1 Pressure/Temperature:01 Temperature °C 00:00.0945 00:19.0175 00:18.0230 20 1.10 Hz
8 70.0 X Quaternion q 00:01.0132 00:18.0954 00:17.0821 1755 98.47 Hz
9 70.1 Y Quaternion q 00:01.0132 00:18.0954 00:17.0821 1755 98.47 Hz
10 70.2 Z Quaternion q 00:01.0132 00:18.0954 00:17.0821 1755 98.47 Hz
11 70.3 W Quaternion q 00:01.0132 00:18.0954 00:17.0821 1755 98.47 Hz
12 59.0 Control Pad Pressure Pressure Pa 00:00.0979 00:18.0910 00:17.0931 180 10.04 Hz
13 59.1 Control Pad Temperature Temperature °C 00:00.0979 00:18.0910 00:17.0931 180 10.04 Hz
14 76.0 Lux Light Ill 00:00.0000 00:18.0737 00:18.0737 71 3.79 Hz
15 76.1 UV Light Index 00:00.0000 00:18.0737 00:18.0737 71 3.79 Hz

The results can be filtered by measurement type:

get_channel_table(doc, ACCELERATION)
  channel name type units start end duration samples rate
0 32.0 X (16g) Acceleration g 00:00.0952 00:19.0012 00:18.0059 7113 393.86 Hz
1 32.1 Y (16g) Acceleration g 00:00.0952 00:19.0012 00:18.0059 7113 393.86 Hz
2 32.2 Z (16g) Acceleration g 00:00.0952 00:19.0012 00:18.0059 7113 393.86 Hz
3 80.0 X (8g) Acceleration g 00:00.0948 00:19.0013 00:18.0064 9070 502.09 Hz
4 80.1 Y (8g) Acceleration g 00:00.0948 00:19.0013 00:18.0064 9070 502.09 Hz
5 80.2 Z (8g) Acceleration g 00:00.0948 00:19.0013 00:18.0064 9070 502.09 Hz

Measurement types can be combined to retrieve more than one:

get_channel_table(doc, ACCELERATION+TEMPERATURE)
  channel name type units start end duration samples rate
0 32.0 X (16g) Acceleration g 00:00.0952 00:19.0012 00:18.0059 7113 393.86 Hz
1 32.1 Y (16g) Acceleration g 00:00.0952 00:19.0012 00:18.0059 7113 393.86 Hz
2 32.2 Z (16g) Acceleration g 00:00.0952 00:19.0012 00:18.0059 7113 393.86 Hz
3 80.0 X (8g) Acceleration g 00:00.0948 00:19.0013 00:18.0064 9070 502.09 Hz
4 80.1 Y (8g) Acceleration g 00:00.0948 00:19.0013 00:18.0064 9070 502.09 Hz
5 80.2 Z (8g) Acceleration g 00:00.0948 00:19.0013 00:18.0064 9070 502.09 Hz
6 36.1 Pressure/Temperature:01 Temperature °C 00:00.0945 00:19.0175 00:18.0230 20 1.10 Hz
7 59.1 Control Pad Temperature Temperature °C 00:00.0979 00:18.0910 00:17.0931 180 10.04 Hz

Information about a specific interval can be retrieved by using the start and/or end arguments. Note that due to different sampling rates, the start and end times for slower channels may not precisely match the specified start or end.

get_channel_table(doc, ACCELERATION+TEMPERATURE, start="0:05", end="0:10")
  channel name type units start end duration samples rate
0 32.0 X (16g) Acceleration g 00:05.0000 00:10.0001 00:05.0000 1969 393.75 Hz
1 32.1 Y (16g) Acceleration g 00:05.0000 00:10.0001 00:05.0000 1969 393.75 Hz
2 32.2 Z (16g) Acceleration g 00:05.0000 00:10.0001 00:05.0000 1969 393.75 Hz
3 80.0 X (8g) Acceleration g 00:05.0000 00:10.0001 00:05.0000 2510 501.98 Hz
4 80.1 Y (8g) Acceleration g 00:05.0000 00:10.0001 00:05.0000 2510 501.98 Hz
5 80.2 Z (8g) Acceleration g 00:05.0000 00:10.0001 00:05.0000 2510 501.98 Hz
6 36.1 Pressure/Temperature:01 Temperature °C 00:04.0954 00:10.0966 00:06.0011 6 1.00 Hz
7 59.1 Control Pad Temperature Temperature °C 00:05.0086 00:10.0095 00:05.0008 50 9.98 Hz

Extracting intervals: endaq.ide.extract_time()#

A portion of an IDE file can be saved to another, new IDE. The source can be a local filename or an opened IDE (from a file or URL).

extract_time("tests/test.ide", "doc_extracted.ide", start="0:05", end="0:10")
extract_time(doc1, "doc1_extracted.ide", start="0:05", end="0:10")

Additional sample IDE recording files#

Here are a number of example IDE files, which may be used with endaq.ide:

file_urls = ['https://info.endaq.com/hubfs/data/surgical-instrument.ide',
             'https://info.endaq.com/hubfs/data/97c3990f-Drive-Home_70-1616632444.ide',
             'https://info.endaq.com/hubfs/data/High-Drop.ide',
             'https://info.endaq.com/hubfs/data/HiTest-Shock.ide',
             'https://info.endaq.com/hubfs/data/Drive-Home_01.ide',
             'https://info.endaq.com/hubfs/data/Tower-of-Terror.ide',
             'https://info.endaq.com/hubfs/data/Punching-Bag.ide',
             'https://info.endaq.com/hubfs/data/Gun-Stock.ide',
             'https://info.endaq.com/hubfs/data/Seat-Base_21.ide',
             'https://info.endaq.com/hubfs/data/Seat-Top_09.ide',
             'https://info.endaq.com/hubfs/data/Bolted.ide',
             'https://info.endaq.com/hubfs/data/Motorcycle-Car-Crash.ide',
             'https://info.endaq.com/hubfs/data/train-passing.ide',
             'https://info.endaq.com/hubfs/data/baseball.ide',
             'https://info.endaq.com/hubfs/data/Clean-Room-VC.ide',
             'https://info.endaq.com/hubfs/data/enDAQ_Cropped.ide',
             'https://info.endaq.com/hubfs/data/Drive-Home_07.ide',
             'https://info.endaq.com/hubfs/data/ford_f150.ide',
             'https://info.endaq.com/hubfs/data/Drive-Home.ide',
             'https://info.endaq.com/hubfs/data/Mining-Data.ide',
             'https://info.endaq.com/hubfs/data/Mide-Airport-Drive-Lexus-Hybrid-Dash-W8.ide']

These can be directly read from endaq.com using endaq.ide.get_doc(), as previously described.