Data Collection, Monitoring and Quality (DMQ) Assurance

The DMQ work packages tackle the issues surrounding high-rate and large-volume data streams, particularly those generated by instruments and sensors.

DMQ4: Online instruments

Overview:

The objective of DMQ4 is to provide online, remote access to pilot working instruments and sensors. According to the architecture of Dart, DMQ4 works between the Readers layer and the Data Source Layer, which likes a bridge between the users and the resource data. Note that the data DMQ4 are interested in is just the raw data in high speed storage, not including the analysed data in tiered storage.

To implement the objective of DMQ4, we deploy the objective toward two directions:

  • Developing visualization tools for data collected from instruments and sensors - to provide the user with a friendly interface to request past data in a particular period or satisfying some constraints.
  • Data processing and data mining for living data streams collected from instruments and sensors - to monitor and analyze especially interesting or emergent data from extensive data streams on-the-fly.

People:

Andrew Newman
Andrew Newman
Sai Sun
Sai Sun
zxf
Xiaofang Zhou

Publications and documents:

Project technical reports:

Presentations:

Outcomes:

Links: