Chapter 2 - Filtering data

In the previous chapter, you learned how to read and print data that is a bit raw. Now, try to select a few columns and handle them properly.

Start with these two columns: Time (time) and Magnitude (mag). After getting the information from these columns, filter and adapt the data. Try formatting the date to Qt types.

There is not much to do for the Magnitude column, as it’s just a floating point number. You could take special care to check if the data is correct. This could be done by filtering the data that follows the condition, “magnitude > 0”, to avoid faulty data or unexpected behavior.

The Date column provides data in UTC format (for example, 2018-12-11T21:14:44.682Z), so you could easily map it to a QDateTime object defining the structure of the string. Additionally, you can adapt the time based on the timezone you are in, using QTimeZone.

The following script filters and formats the CSV data as described earlier:

 1from __future__ import annotations
 2
 3import argparse
 4import pandas as pd
 5
 6from PySide6.QtCore import QDateTime, QTimeZone
 7
 8
 9def transform_date(utc, timezone=None):
10    utc_fmt = "yyyy-MM-ddTHH:mm:ss.zzzZ"
11    new_date = QDateTime().fromString(utc, utc_fmt)
12    if timezone:
13        new_date.setTimeZone(timezone)
14    return new_date
15
16
17def read_data(fname):
18    # Read the CSV content
19    df = pd.read_csv(fname)
20
21    # Remove wrong magnitudes
22    df = df.drop(df[df.mag < 0].index)
23    magnitudes = df["mag"]
24
25    # My local timezone
26    timezone = QTimeZone(b"Europe/Berlin")
27
28    # Get timestamp transformed to our timezone
29    times = df["time"].apply(lambda x: transform_date(x, timezone))
30
31    return times, magnitudes
32
33
34if __name__ == "__main__":
35    options = argparse.ArgumentParser()
36    options.add_argument("-f", "--file", type=str, required=True)
37    args = options.parse_args()
38    data = read_data(args.file)
39    print(data)
40

Now that you have a tuple of QDateTime and float data, try improving the output further. That’s what you’ll learn in the following chapters.