{ "nbformat": 4, "nbformat_minor": 0, "metadata": { "colab": { "name": "enDAQ-Python-Quick-Start.ipynb", "provenance": [], "collapsed_sections": [], "toc_visible": true }, "kernelspec": { "name": "python3", "display_name": "Python 3" }, "language_info": { "name": "python" } }, "cells": [ { "cell_type": "code", "execution_count": null, "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "ai4iXgA7v9B3", "outputId": "70aa031b-c56c-4750-eed4-44ad9e9866fc" }, "outputs": [ { "output_type": "stream", "name": "stdout", "text": [ " Installing build dependencies ... \u001B[?25l\u001B[?25hdone\n", " Getting requirements to build wheel ... \u001B[?25l\u001B[?25hdone\n", " Preparing wheel metadata ... \u001B[?25l\u001B[?25hdone\n", "\u001B[K |████████████████████████████████| 63 kB 1.1 MB/s \n", "\u001B[K |████████████████████████████████| 93 kB 879 kB/s \n", "\u001B[K |████████████████████████████████| 38.1 MB 1.3 MB/s \n", "\u001B[K |████████████████████████████████| 83 kB 1.2 MB/s \n", "\u001B[?25h Building wheel for endaq (PEP 517) ... \u001B[?25l\u001B[?25hdone\n", "\u001B[31mERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.\n", "google-colab 1.0.0 requires requests~=2.23.0, but you have requests 2.27.1 which is incompatible.\n", "datascience 0.10.6 requires folium==0.2.1, but you have folium 0.8.3 which is incompatible.\n", "albumentations 0.1.12 requires imgaug<0.2.7,>=0.2.5, but you have imgaug 0.2.9 which is incompatible.\u001B[0m\n" ] } ], "source": [ "!pip install -q git+https://github.com/MideTechnology/endaq-python.git@development\n", "exit() #forces a restart, it will make Colab crash - that is okay! just go on to run all the cells below\n", " #Note that this is only needed in colab and not if running locally" ] }, { "cell_type": "markdown", "source": [ "# Quick Start Guide\n", "\n", "## Introduction\n", "This example script will use the enDAQ Python library to do the following:\n", "\n", "* Load open-source libraries\n", "* Load data directly from an IDE file recorded from an enDAQ sensor\n", " * Provide meta data about the sensor which recorded the data\n", " * Summary the contents of the file as a table of all sensor channels\n", " * Summarize all data in a dashboard\n", "* Get the accelerometer data\n", " * Apply a high pass filter\n", " * Plot the full time history\n", " * Plot the time history around the peak\n", " * Plot the time history in a specific time range\n", "* Shock & Vibration Analysis\n", " * Linear PSD\n", " * Octave spaced PSD\n", " * FFT\n", " * Shock response spectrum\n", "\n" ], "metadata": { "id": "2yLrZ1qpwWQi" } }, { "cell_type": "markdown", "source": [ "## Load Libraries" ], "metadata": { "id": "w1FzP7qQxp2W" } }, { "cell_type": "code", "source": [ "import endaq\n", "\n", "endaq.plot.utilities.set_theme('endaq_light')\n", "\n", "import plotly.express as px\n", "import pandas as pd\n", "import numpy as np\n", "import scipy" ], "metadata": { "id": "31QXc4iSwUrY" }, "execution_count": null, "outputs": [] }, { "cell_type": "markdown", "source": [ "## Load Data\n", "\n", "Using the [endaq.ide.get_doc function](https://docs.endaq.com/en/development/endaq/ide.html#endaq.ide.get_doc), load in the contents of an IDE file. Note that this doesn't load all data into memory yet." ], "metadata": { "id": "LergWzEMxdRw" } }, { "cell_type": "code", "source": [ "doc = endaq.ide.get_doc('https://info.endaq.com/hubfs/data/Mining-Hammer/LOC__3__DAQ41551_11_01_02.ide')" ], "metadata": { "id": "N4cWY6Eux1MA" }, "execution_count": null, "outputs": [] }, { "cell_type": "markdown", "source": [ "### Display meta data" ], "metadata": { "id": "FpowMKFyxzJl" } }, { "cell_type": "code", "source": [ "print(f\"Serial Number: {doc.recorderInfo['RecorderSerial']}\")\n", "print(f\"Part Number: {doc.recorderInfo['PartNumber']}\")\n", "print(f\"Recording Finished at {pd.to_datetime(doc.lastUtcTime,unit='s')}\")" ], "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "SubLdBG8yQQo", "outputId": "4f87e5ad-0ca1-4618-f98c-9e44374ac4ca" }, "execution_count": null, "outputs": [ { "output_type": "stream", "name": "stdout", "text": [ "Serial Number: 9680\n", "Part Number: S3-E2000D40\n", "Recording Finished at 2021-03-25 03:32:32\n" ] } ] }, { "cell_type": "markdown", "source": [ "### Table of file contents\n", "This uses the function [endaq.ide.get_channel_table](https://docs.endaq.com/en/development/endaq/ide.html#endaq.ide.get_channel_table)." ], "metadata": { "id": "iNgnSQzLzVfG" } }, { "cell_type": "code", "source": [ "endaq.ide.get_channel_table(doc)" ], "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 394 }, "id": "9W2HaZKsy2VE", "outputId": "cb8d72fa-4856-4cbc-af7b-33f7c55e8b2f" }, "execution_count": null, "outputs": [ { "output_type": "execute_result", "data": { "text/html": [ "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
 channelnametypeunitsstartenddurationsamplesrate
08.0X (2000g)Accelerationg34:01.086834:15.056100:13.069327386720000.07 Hz
18.1Y (2000g)Accelerationg34:01.086834:15.056100:13.069327386720000.07 Hz
28.2Z (2000g)Accelerationg34:01.086834:15.056100:13.069327386720000.07 Hz
380.0X (40g)Accelerationg34:01.071734:15.053100:13.0814555104018.33 Hz
480.1Y (40g)Accelerationg34:01.071734:15.053100:13.0814555104018.33 Hz
580.2Z (40g)Accelerationg34:01.071734:15.053100:13.0814555104018.33 Hz
636.0Pressure/Temperature:00PressurePa33:47.010534:12.013000:25.0025261.04 Hz
736.1Pressure/Temperature:01Temperature°C33:47.010534:12.013000:25.0025261.04 Hz
847.0XRotationdps34:00.094734:14.063300:13.06852765202.04 Hz
947.1YRotationdps34:00.094734:14.063300:13.06852765202.04 Hz
1047.2ZRotationdps34:00.094734:14.063300:13.06852765202.04 Hz
\n" ], "text/plain": [ "" ] }, "metadata": {}, "execution_count": 4 } ] }, { "cell_type": "markdown", "source": [ "### Dashboard of all data\n", "Note this requires first loading in all data into memory. Then it will display bars of all data plotted from min to max." ], "metadata": { "id": "LBH1fGRszlId" } }, { "cell_type": "code", "source": [ "channel_dict = {doc.channels[ch].name: endaq.ide.to_pandas(doc.channels[ch]) for ch in doc.channels}\n", "fig = endaq.plot.dashboards.rolling_enveloped_dashboard(channel_dict, plot_as_bars=True, num_rows=1, num_cols=None)\n", "fig.show()" ], "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 437 }, "id": "aZicwV4Oxyof", "outputId": "974003e5-6594-4d81-8412-7c8f5ddb3a03" }, "execution_count": null, "outputs": [ { "output_type": "display_data", "data": { "text/html": [ "\n", "\n", "\n", "
\n", "
\n", "\n", "" ] }, "metadata": {} } ] }, { "cell_type": "markdown", "source": [ "## Get accelerometer data" ], "metadata": { "id": "Rbtr6sI-0hfn" } }, { "cell_type": "code", "source": [ "accel = endaq.ide.get_primary_sensor_data(doc=doc,measurement_type='accel',time_mode='seconds')\n", "accel" ], "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 455 }, "id": "i4yTO_ak0tuo", "outputId": "13ffa11a-168b-48e9-b741-9da02aa69998" }, "execution_count": null, "outputs": [ { "output_type": "execute_result", "data": { "text/html": [ "\n", "
\n", "
\n", "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
X (2000g)Y (2000g)Z (2000g)
timestamp
2041.868560-24.04686114.828584-9.588154
2041.868610-7.20580114.828584-11.630245
2041.86866014.49659519.59430620.532687
2041.86871055.12348013.24001077.541059
2041.86876083.9442621.76697684.518203
............
2055.56165950.95662013.9460435.387179
2055.56170962.06824714.8285847.939793
2055.56175940.71308914.8285845.217005
2055.5618092.8641116.7092063.855611
2055.561859-21.616192-9.1765333.004740
\n", "

273867 rows × 3 columns

\n", "
\n", " \n", " \n", " \n", "\n", " \n", "
\n", "
\n", " " ], "text/plain": [ " X (2000g) Y (2000g) Z (2000g)\n", "timestamp \n", "2041.868560 -24.046861 14.828584 -9.588154\n", "2041.868610 -7.205801 14.828584 -11.630245\n", "2041.868660 14.496595 19.594306 20.532687\n", "2041.868710 55.123480 13.240010 77.541059\n", "2041.868760 83.944262 1.766976 84.518203\n", "... ... ... ...\n", "2055.561659 50.956620 13.946043 5.387179\n", "2055.561709 62.068247 14.828584 7.939793\n", "2055.561759 40.713089 14.828584 5.217005\n", "2055.561809 2.864111 6.709206 3.855611\n", "2055.561859 -21.616192 -9.176533 3.004740\n", "\n", "[273867 rows x 3 columns]" ] }, "metadata": {}, "execution_count": 7 } ] }, { "cell_type": "markdown", "source": [ "### Apply High Pass Filter\n", "Filter away the DC offset, there are more filtering options at [endaq.calc.filters](https://docs.endaq.com/en/development/endaq/calc.html#module-endaq.calc.filters)." ], "metadata": { "id": "IqCUaicN1QSC" } }, { "cell_type": "code", "source": [ "accel = endaq.calc.filters.butterworth(accel,low_cutoff=1)" ], "metadata": { "id": "UBt8YspV1BhQ" }, "execution_count": null, "outputs": [] }, { "cell_type": "markdown", "source": [ "### Plot Full Time History\n", "[endaq.plot.rolling_min_max_envelope()](https://docs.endaq.com/en/development/endaq/plot.html#endaq.plot.rolling_min_max_envelope) will reduce a large time history into a bar chart that plots from the min to the max so that it appears identical to the full time history plot yet is fast to generate and responsive." ], "metadata": { "id": "EChu4oiq1bkm" } }, { "cell_type": "code", "source": [ "fig_full_accel = endaq.plot.rolling_min_max_envelope(\n", " accel,\n", " desired_num_points=1000,\n", " plot_as_bars=True,\n", " opacity=0.7\n", ").update_layout(\n", " yaxis_title_text='Acceleration (g)',\n", " xaxis_title_text='Time (s)',\n", " title_text='Full Time History of Acceleration Data'\n", ")\n", "fig_full_accel.show()" ], "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 542 }, "id": "W8a68bAZ1W1P", "outputId": "0975955e-84f9-411e-e857-2e7d288f3515" }, "execution_count": null, "outputs": [ { "output_type": "display_data", "data": { "text/html": [ "\n", "\n", "\n", "
\n", "
\n", "\n", "" ] }, "metadata": {} } ] }, { "cell_type": "markdown", "source": [ "### Plot Time History Around Peak\n", "[endaq.plot.around_peak()](https://docs.endaq.com/en/development/endaq/plot.html#endaq.plot.around_peak) finds and plots around the maximum value." ], "metadata": { "id": "IpntI8vH11CY" } }, { "cell_type": "code", "source": [ "fig_peak = endaq.plot.around_peak(accel, num=1000).update_layout(\n", " yaxis_title_text='Acceleration (g)',\n", " xaxis_title_text='Time (s)',\n", " title_text='Time History Around Peak',\n", " legend_title_text=''\n", ")\n", "fig_peak.show()" ], "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 542 }, "id": "AYU5DvQy1i4I", "outputId": "f53167c4-b535-48fc-b4ce-a9007e31e05a" }, "execution_count": null, "outputs": [ { "output_type": "display_data", "data": { "text/html": [ "\n", "\n", "\n", "
\n", "
\n", "\n", "" ] }, "metadata": {} } ] }, { "cell_type": "markdown", "source": [ "### Time History at Specific Time" ], "metadata": { "id": "KRBmXKpz2JH3" } }, { "cell_type": "code", "source": [ "fig_time = px.line(accel[2044.6:2044.7]).update_layout(\n", " yaxis_title_text='Acceleration (g)',\n", " xaxis_title_text='Time (s)',\n", " title_text='Time History at Specific Time',\n", " legend_title_text=''\n", ")\n", "fig_time.show()" ], "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 542 }, "id": "FvLw-PAM1_rI", "outputId": "748e5d8e-aa88-4fd3-8d3e-f73be2371f4f" }, "execution_count": null, "outputs": [ { "output_type": "display_data", "data": { "text/html": [ "\n", "\n", "\n", "
\n", "
\n", "\n", "" ] }, "metadata": {} } ] }, { "cell_type": "markdown", "source": [ "## Analysis" ], "metadata": { "id": "P47S_tkO2s-u" } }, { "cell_type": "markdown", "source": [ "### Linear PSD\n", "Calculate the PSD using [endaq.calc.psd.welch()](https://docs.endaq.com/en/development/endaq/calc.html#endaq.calc.psd.welch)" ], "metadata": { "id": "eqCuawCy3EDr" } }, { "cell_type": "code", "source": [ "#Calculate PSD\n", "psd = endaq.calc.psd.welch(accel, bin_width=2)\n", "\n", "fig_psd = px.line(\n", " psd,\n", " log_x=True,\n", " log_y=True\n", ").update_layout(\n", " xaxis_title='Frequency (Hz)',\n", " yaxis_title='Acceleration (g^2/Hz)',\n", " title_text='PSD with 2 Hz Bin Width',\n", " legend_title_text=''\n", ")\n", "\n", "fig_psd.show()" ], "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 542 }, "id": "VcSWUJHJ2T44", "outputId": "58bd45e1-b338-47e1-c262-74b11ae59896" }, "execution_count": null, "outputs": [ { "output_type": "display_data", "data": { "text/html": [ "\n", "\n", "\n", "
\n", "
\n", "\n", "" ] }, "metadata": {} } ] }, { "cell_type": "markdown", "source": [ "### Octave PSD\n", "\n", "Now convert to octave using [endaq.calc.psd.to_octave()](https://docs.endaq.com/en/development/endaq/calc.html#endaq.calc.psd.to_octave)" ], "metadata": { "id": "UpQuiJNR3GAI" } }, { "cell_type": "code", "source": [ "#Calculate PSD with a finer bin width to convert to octave\n", "psd = endaq.calc.psd.welch(accel, bin_width=0.1)\n", "\n", "#Convert to Octave\n", "oct_psd = endaq.calc.psd.to_octave(psd,octave_bins=6)\n", "\n", "fig_psd_oct = px.line(\n", " oct_psd,\n", " log_x=True,\n", " log_y=True\n", ").update_layout(\n", " xaxis_title='Frequency (Hz)',\n", " yaxis_title='Acceleration (g^2/Hz)',\n", " title_text='1/6 Octave PSD',\n", " legend_title_text=''\n", ")\n", "\n", "fig_psd_oct.show()" ], "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 612 }, "id": "3fugBhBJ27cI", "outputId": "87730694-f05b-4691-b278-c9aeeb6194f3" }, "execution_count": null, "outputs": [ { "output_type": "stream", "name": "stderr", "text": [ "/usr/local/lib/python3.7/dist-packages/endaq/calc/psd.py:172: RuntimeWarning:\n", "\n", "empty frequency bins in re-binned PSD; original PSD's frequency spacing is too coarse\n", "\n" ] }, { "output_type": "display_data", "data": { "text/html": [ "\n", "\n", "\n", "
\n", "
\n", "\n", "" ] }, "metadata": {} } ] }, { "cell_type": "markdown", "source": [ "### FFT\n", "See [endaq.calc.fft](https://docs.endaq.com/en/development/endaq/calc.html#module-endaq.calc.fft) for a full list of FFT functions." ], "metadata": { "id": "K8QYoQA333UU" } }, { "cell_type": "code", "source": [ "#Calculate FFT\n", "fft = endaq.calc.fft.rfft(accel[2044.6:2044.7])\n", "\n", "#Plot\n", "fig_fft = px.line(fft).update_layout(\n", " xaxis_title='Frequency (Hz)',\n", " yaxis_title='Acceleration (g)',\n", " title_text='FFT of Time Range 2044.6 to 2044.7 Seconds',\n", " legend_title_text=''\n", ")\n", "\n", "fig_fft.show()" ], "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 542 }, "id": "EmUQ_MSj3rfB", "outputId": "2c2a9323-d1ee-4117-b170-044d3bd9ee78" }, "execution_count": null, "outputs": [ { "output_type": "display_data", "data": { "text/html": [ "\n", "\n", "\n", "
\n", "
\n", "\n", "" ] }, "metadata": {} } ] }, { "cell_type": "markdown", "source": [ "### Shock Response Spectrum\n", "This uses [endaq.calc.utils.logfreqs()](https://docs.endaq.com/en/development/endaq/calc.html#endaq.calc.utils.logfreqs) to define log spaced frequencies to use, then passes those to [endaq.calc.shock.shock_spectrum()](https://docs.endaq.com/en/development/endaq/calc.html#endaq.calc.shock.shock_spectrum) along with the time history data to compute the shock spectrum." ], "metadata": { "id": "zPfOkD5Z4zpj" } }, { "cell_type": "code", "source": [ "#Calculate for reduced time series\n", "accel_zoomed = accel[2044.6:2044.7]\n", "\n", "#Define frequencies to calculate responses at\n", "freqs = endaq.calc.utils.logfreqs(accel_zoomed, bins_per_octave=12, init_freq=0.5)\n", "\n", "#Calculate pseudo velocity shock spectrum\n", "pvss = endaq.calc.shock.shock_spectrum(accel_zoomed, freqs=freqs, damp=0.05, mode='pvss')\n", "\n", "#Convert to in/s (currently in g/s)\n", "pvss = pvss * 9.81 * 39.37\n", "\n", "#Plot\n", "fig_pvss = px.line(\n", " pvss,\n", " log_x=True,\n", " log_y=True\n", ").update_layout(\n", " xaxis_title='Natural Frequency (Hz)',\n", " yaxis_title='Peak Pseudo Velocity (in/s)',\n", " legend_title_text='',\n", " title_text='Psuedo Velocity Shock Spectrum'\n", ")\n", "\n", "fig_pvss.show()" ], "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 612 }, "id": "NIR52PSJ4MnJ", "outputId": "7ec29c17-4771-4439-bf0d-6909447317ad", "pycharm": { "name": "#%%\n" } }, "execution_count": null, "outputs": [ { "output_type": "stream", "name": "stderr", "text": [ "/usr/local/lib/python3.7/dist-packages/endaq/calc/utils.py:61: RuntimeWarning:\n", "\n", "the data's duration is too short to accurately represent an initial frequency of 0.500 Hz\n", "\n" ] }, { "output_type": "display_data", "data": { "text/html": [ "\n", "\n", "\n", "
\n", "
\n", "\n", "" ] }, "metadata": {} } ] } ] }