[ ]:

!pip install -q git+https://github.com/MideTechnology/endaq-python.git@development
exit()  #forces a restart, it will make Colab crash - that is okay! just go on to run all the cells below
        #Note that this is only needed in colab and not if running locally

  Installing build dependencies ... done
  Getting requirements to build wheel ... done
    Preparing wheel metadata ... done
     |████████████████████████████████| 63 kB 1.1 MB/s
     |████████████████████████████████| 93 kB 879 kB/s
     |████████████████████████████████| 38.1 MB 1.3 MB/s
     |████████████████████████████████| 83 kB 1.2 MB/s
  Building wheel for endaq (PEP 517) ... done
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
google-colab 1.0.0 requires requests~=2.23.0, but you have requests 2.27.1 which is incompatible.
datascience 0.10.6 requires folium==0.2.1, but you have folium 0.8.3 which is incompatible.
albumentations 0.1.12 requires imgaug<0.2.7,>=0.2.5, but you have imgaug 0.2.9 which is incompatible.

Quick Start Guide

Introduction

This example script will use the enDAQ Python library to do the following:

  • Load open-source libraries

  • Load data directly from an IDE file recorded from an enDAQ sensor

    • Provide meta data about the sensor which recorded the data

    • Summary the contents of the file as a table of all sensor channels

    • Summarize all data in a dashboard

  • Get the accelerometer data

    • Apply a high pass filter

    • Plot the full time history

    • Plot the time history around the peak

    • Plot the time history in a specific time range

  • Shock & Vibration Analysis

    • Linear PSD

    • Octave spaced PSD

    • FFT

    • Shock response spectrum

Load Libraries

[ ]:

import endaq

endaq.plot.utilities.set_theme('endaq_light')

import plotly.express as px
import pandas as pd
import numpy as np
import scipy

Load Data

Using the endaq.ide.get_doc function, load in the contents of an IDE file. Note that this doesn’t load all data into memory yet.

[ ]:

doc = endaq.ide.get_doc('https://info.endaq.com/hubfs/data/Mining-Hammer/LOC__3__DAQ41551_11_01_02.ide')

Display meta data

[ ]:

print(f"Serial Number: {doc.recorderInfo['RecorderSerial']}")
print(f"Part Number: {doc.recorderInfo['PartNumber']}")
print(f"Recording Finished at {pd.to_datetime(doc.lastUtcTime,unit='s')}")

Serial Number: 9680
Part Number: S3-E2000D40
Recording Finished at 2021-03-25 03:32:32

Table of file contents

This uses the function endaq.ide.get_channel_table.

[ ]:

endaq.ide.get_channel_table(doc)
  channel name type units start end duration samples rate
0 8.0 X (2000g) Acceleration g 34:01.0868 34:15.0561 00:13.0693 273867 20000.07 Hz
1 8.1 Y (2000g) Acceleration g 34:01.0868 34:15.0561 00:13.0693 273867 20000.07 Hz
2 8.2 Z (2000g) Acceleration g 34:01.0868 34:15.0561 00:13.0693 273867 20000.07 Hz
3 80.0 X (40g) Acceleration g 34:01.0717 34:15.0531 00:13.0814 55510 4018.33 Hz
4 80.1 Y (40g) Acceleration g 34:01.0717 34:15.0531 00:13.0814 55510 4018.33 Hz
5 80.2 Z (40g) Acceleration g 34:01.0717 34:15.0531 00:13.0814 55510 4018.33 Hz
6 36.0 Pressure/Temperature:00 Pressure Pa 33:47.0105 34:12.0130 00:25.0025 26 1.04 Hz
7 36.1 Pressure/Temperature:01 Temperature °C 33:47.0105 34:12.0130 00:25.0025 26 1.04 Hz
8 47.0 X Rotation dps 34:00.0947 34:14.0633 00:13.0685 2765 202.04 Hz
9 47.1 Y Rotation dps 34:00.0947 34:14.0633 00:13.0685 2765 202.04 Hz
10 47.2 Z Rotation dps 34:00.0947 34:14.0633 00:13.0685 2765 202.04 Hz

Dashboard of all data

Note this requires first loading in all data into memory. Then it will display bars of all data plotted from min to max.

[ ]:

channel_dict = {doc.channels[ch].name: endaq.ide.to_pandas(doc.channels[ch]) for ch in doc.channels}
fig = endaq.plot.dashboards.rolling_enveloped_dashboard(channel_dict, plot_as_bars=True, num_rows=1, num_cols=None)
fig.show()
04:06:35Mar 25, 202104:06:4004:06:45−400−200020040060004:06:35Mar 25, 202104:06:4004:06:45−150−100−5005010004:06:35Mar 25, 202104:06:4004:06:45−400−200020040004:06:35Mar 25, 202104:06:4004:06:45−40−200204004:06:35Mar 25, 202104:06:4004:06:45−40−2002004:06:35Mar 25, 202104:06:4004:06:45−40−200204004:06:20Mar 25, 202104:06:3004:06:40105.3k105.4k105.5k105.6k105.7k105.8k04:06:20Mar 25, 202104:06:3004:06:4033.433.4533.533.5504:06:35Mar 25, 202104:06:4004:06:45−500050004:06:35Mar 25, 202104:06:4004:06:45−200020040004:06:35Mar 25, 202104:06:4004:06:45−600−400−2000200400
2000g PE Acceleration X (2000g)2000g PE Acceleration Y (2000g)2000g PE Acceleration Z (2000g)40g DC Acceleration X (40g)40g DC Acceleration Y (40g)40g DC Acceleration Z (40g)Pressure/Temperature Pressure/Temperature:00Pressure/Temperature Pressure/Temperature:01Rotation XRotation YRotation Z

Get accelerometer data

[ ]:

accel = endaq.ide.get_primary_sensor_data(doc=doc,measurement_type='accel',time_mode='seconds')
accel
X (2000g) Y (2000g) Z (2000g)
timestamp
2041.868560 -24.046861 14.828584 -9.588154
2041.868610 -7.205801 14.828584 -11.630245
2041.868660 14.496595 19.594306 20.532687
2041.868710 55.123480 13.240010 77.541059
2041.868760 83.944262 1.766976 84.518203
... ... ... ...
2055.561659 50.956620 13.946043 5.387179
2055.561709 62.068247 14.828584 7.939793
2055.561759 40.713089 14.828584 5.217005
2055.561809 2.864111 6.709206 3.855611
2055.561859 -21.616192 -9.176533 3.004740

273867 rows × 3 columns

Apply High Pass Filter

Filter away the DC offset, there are more filtering options at endaq.calc.filters.

[ ]:

accel = endaq.calc.filters.butterworth(accel,low_cutoff=1)

Plot Full Time History

endaq.plot.rolling_min_max_envelope() will reduce a large time history into a bar chart that plots from the min to the max so that it appears identical to the full time history plot yet is fast to generate and responsive.

[ ]:

fig_full_accel = endaq.plot.rolling_min_max_envelope(
    accel,
    desired_num_points=1000,
    plot_as_bars=True,
    opacity=0.7
).update_layout(
    yaxis_title_text='Acceleration (g)',
    xaxis_title_text='Time (s)',
    title_text='Full Time History of Acceleration Data'
)
fig_full_accel.show()
2042204420462048205020522054−400−2000200400600
X (2000g)Y (2000g)Z (2000g)Full Time History of Acceleration DataTime (s)Acceleration (g)

Plot Time History Around Peak

endaq.plot.around_peak() finds and plots around the maximum value.

[ ]:

fig_peak = endaq.plot.around_peak(accel, num=1000).update_layout(
    yaxis_title_text='Acceleration (g)',
    xaxis_title_text='Time (s)',
    title_text='Time History Around Peak',
    legend_title_text=''
)
fig_peak.show()
2,044.582,044.5852,044.592,044.5952,044.62,044.6052,044.612,044.6152,044.622,044.625−400−2000200400600
X (2000g)Y (2000g)Z (2000g)Time History Around PeakTime (s)Acceleration (g)

Time History at Specific Time

[ ]:

fig_time = px.line(accel[2044.6:2044.7]).update_layout(
    yaxis_title_text='Acceleration (g)',
    xaxis_title_text='Time (s)',
    title_text='Time History at Specific Time',
    legend_title_text=''
)
fig_time.show()
2,044.612,044.622,044.632,044.642,044.652,044.662,044.672,044.682,044.69−400−2000200400600
X (2000g)Y (2000g)Z (2000g)Time History at Specific TimeTime (s)Acceleration (g)

Analysis

Linear PSD

Calculate the PSD using endaq.calc.psd.welch()

[ ]:

#Calculate PSD
psd = endaq.calc.psd.welch(accel, bin_width=2)

fig_psd = px.line(
    psd,
    log_x=True,
    log_y=True
).update_layout(
    xaxis_title='Frequency (Hz)',
    yaxis_title='Acceleration (g^2/Hz)',
    title_text='PSD with 2 Hz Bin Width',
    legend_title_text=''
)

fig_psd.show()
2510251002510002510k10μ100μ0.0010.010.1110
X (2000g)Y (2000g)Z (2000g)PSD with 2 Hz Bin WidthFrequency (Hz)Acceleration (g^2/Hz)

Octave PSD

Now convert to octave using endaq.calc.psd.to_octave()

[ ]:

#Calculate PSD with a finer bin width to convert to octave
psd = endaq.calc.psd.welch(accel, bin_width=0.1)

#Convert to Octave
oct_psd = endaq.calc.psd.to_octave(psd,octave_bins=6)

fig_psd_oct = px.line(
    oct_psd,
    log_x=True,
    log_y=True
).update_layout(
    xaxis_title='Frequency (Hz)',
    yaxis_title='Acceleration (g^2/Hz)',
    title_text='1/6 Octave PSD',
    legend_title_text=''
)

fig_psd_oct.show()

/usr/local/lib/python3.7/dist-packages/endaq/calc/psd.py:172: RuntimeWarning:

empty frequency bins in re-binned PSD; original PSD's frequency spacing is too coarse
12510251002510002510k10μ100μ0.0010.010.1110
X (2000g)Y (2000g)Z (2000g)1/6 Octave PSDFrequency (Hz)Acceleration (g^2/Hz)

FFT

See endaq.calc.fft for a full list of FFT functions.

[ ]:

#Calculate FFT
fft = endaq.calc.fft.rfft(accel[2044.6:2044.7])

#Plot
fig_fft = px.line(fft).update_layout(
    xaxis_title='Frequency (Hz)',
    yaxis_title='Acceleration (g)',
    title_text='FFT of Time Range 2044.6 to 2044.7 Seconds',
    legend_title_text=''
)

fig_fft.show()
01k2k3k4k5k6k7k8k9k10k010203040
X (2000g)Y (2000g)Z (2000g)FFT of Time Range 2044.6 to 2044.7 SecondsFrequency (Hz)Acceleration (g)

Shock Response Spectrum

This uses endaq.calc.utils.logfreqs() to define log spaced frequencies to use, then passes those to endaq.calc.shock.shock_spectrum() along with the time history data to compute the shock spectrum.

[ ]:

#Calculate for reduced time series
accel_zoomed = accel[2044.6:2044.7]

#Define frequencies to calculate responses at
freqs = endaq.calc.utils.logfreqs(accel_zoomed, bins_per_octave=12, init_freq=0.5)

#Calculate pseudo velocity shock spectrum
pvss = endaq.calc.shock.shock_spectrum(accel_zoomed, freqs=freqs, damp=0.05, mode='pvss')

#Convert to in/s (currently in g/s)
pvss = pvss * 9.81 * 39.37

#Plot
fig_pvss = px.line(
    pvss,
    log_x=True,
    log_y=True
).update_layout(
    xaxis_title='Natural Frequency (Hz)',
    yaxis_title='Peak Pseudo Velocity (in/s)',
    legend_title_text='',
    title_text='Psuedo Velocity Shock Spectrum'
)

fig_pvss.show()

/usr/local/lib/python3.7/dist-packages/endaq/calc/utils.py:61: RuntimeWarning:

the data's duration is too short to accurately represent an initial frequency of 0.500 Hz
51251025100251000257891234567891023456789100
X (2000g)Y (2000g)Z (2000g)Psuedo Velocity Shock SpectrumNatural Frequency (Hz)Peak Pseudo Velocity (in/s)

previous

enDAQ Custom Analysis