{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Quick start\n", "\n", "This quick start present data processing using Terra15 DAS recording velocity. It can easily adapted for Febus DAS recording strain rate.\n", "\n", "The processing is taking advantage of [dask+xarray framework.](https://docs.xarray.dev/en/stable/user-guide/dask.html) " ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [], "source": [ "# specific das_ice function\n", "import das_ice.io as di_io\n", "import das_ice.signal.filter as di_filter\n", "import das_ice.processes as di_p\n", "import das_ice.plot as di_plt\n", "\n", "# classic librairy\n", "import xarray as xr\n", "import pandas as pd\n", "import numpy as np\n", "import matplotlib.pyplot as plt\n", "from matplotlib.colors import LogNorm\n", "from tqdm import trange\n", "%matplotlib inline" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "'http://127.0.0.1:8787/status'" ] }, "execution_count": 2, "metadata": {}, "output_type": "execute_result" } ], "source": [ "from dask.distributed import Client,LocalCluster\n", "\n", "cluster = LocalCluster(n_workers=8)\n", "client = Client(cluster)\n", "client.dashboard_link" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Loading data" ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [], "source": [ "ds=xr.Dataset()\n", "ds['velocity']=di_io.dask_Terra15('*.hdf5', chunks=\"auto\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The program reads HDF5 files without loading them entirely into memory. This allows for efficient selection of **time** and **distance** ranges before processing. One benefit of using Dask is its ability to handle large datasets that cannot fit into memory. Additionally, Dask enables distributed parallel computing, ensuring efficient execution across multiple processors or machines." ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
<xarray.DataArray 'velocity' (time: 1236726, distance: 618)> Size: 3GB\n", "dask.array<concatenate, shape=(1236726, 618), dtype=float32, chunksize=(54100, 618), chunktype=numpy.ndarray>\n", "Coordinates:\n", " * distance (distance) float64 5kB 674.6 678.7 682.8 ... 3.19e+03 3.194e+03\n", " * time (time) datetime64[ns] 10MB 2024-08-29T05:04:48.597790 ... 2024-...\n", "Attributes:\n", " client_fn_applied: \n", " dT: 0.000147915811144255\n", " dx: 4.083809535485629\n", " frame_shape: [1082 618]\n", " nT: 1082\n", " nx: 618\n", " recorder_id: T311542\n", " trigger_start_time: -1.0
<xarray.DataArray 'velocity' (time: 68733, distance: 47)> Size: 13MB\n", "dask.array<getitem, shape=(68733, 47), dtype=float32, chunksize=(9017, 47), chunktype=numpy.ndarray>\n", "Coordinates:\n", " * distance (distance) float64 376B 2.264 3.88 6.348 ... 92.11 93.72 96.19\n", " * time (time) datetime64[ns] 550kB 2024-08-29T05:05:30.000017080 ... 2...\n", "Attributes:\n", " client_fn_applied: \n", " dT: 0.000147915811144255\n", " dx: 4.083809535485629\n", " frame_shape: [1082 618]\n", " nT: 1082\n", " nx: 618\n", " recorder_id: T311542\n", " trigger_start_time: -1.0
<xarray.DataArray 'velocity' (time: 68733, distance: 47)> Size: 13MB\n", "dask.array<transpose, shape=(68733, 47), dtype=float32, chunksize=(68733, 47), chunktype=numpy.ndarray>\n", "Coordinates:\n", " * distance (distance) float64 376B 2.264 3.88 6.348 ... 92.11 93.72 96.19\n", " * time (time) datetime64[ns] 550kB 2024-08-29T05:05:30.000017080 ... 2...
<xarray.DataArray 'velocity' (time: 68733, distance: 47)> Size: 13MB\n", "dask.array<transpose, shape=(68733, 47), dtype=float32, chunksize=(68733, 47), chunktype=numpy.ndarray>\n", "Coordinates:\n", " * distance (distance) float64 376B 2.264 3.88 6.348 ... 92.11 93.72 96.19\n", " * time (time) datetime64[ns] 550kB 2024-08-29T05:05:30.000017080 ... 2...