INDUSTRY & IOT
IoT data processing: from MQTT streams to usable data
IoT data processing with TIMi uses Anatella to subscribe to MQTT brokers, transform sensor data as it arrives or in batches, and publish results back, on one or two standard servers.
MQTT out of the box
Subscribe and publish actions for IoT brokers.
One or two servers
No cluster for typical IoT volumes.
Ready for models
Sensor data prepared for predictive modeling.
HOW IT WORKS
How IoT data processing works.
Three steps, in one engine, on a standard server with no cluster.
1. Subscribe to the stream
An MQTT subscribe action connects to your broker and receives messages as they are published.
2. Transform the data
Row-by-row actions process each message as it arrives. For heavier steps, raw data is stored and processed in batches.
3. Publish or store
Results are published back to the broker, or written to files, databases, Kibella or scoring pipelines.
AnatellaGETTING IT RIGHT
Choices that shape an IoT pipeline.
Decide what must happen in real time and what can wait for a batch.
Real time has constraints
A streaming flow only supports actions that work row by row. An action that must see all rows first, such as a sort, cannot run in streaming mode.
Combine streaming and batch
Quick reactions run in streaming mode. Aggregations and model training run on stored data in batch.
Plan the data volume
Raw sensor data grows quickly. Decide what to keep raw and what to aggregate, starting from the questions you want to answer.
TRANSPARENCY
A pipeline you can follow.
An IoT pipeline is a graph of boxes from the broker to the target. Each step is visible, which helps when a measure looks wrong and someone needs to find where it changed.
For engineers
See each step from message to result.
For debugging
Row counters show whether messages flow through every step.
For security
Data stays on your servers, with no cloud dependency.
DATA
What IoT data can be processed.
Anatella connects to more than 80 sources, so this data can be gathered without prior custom development.
Sensor messages
Measures published on an MQTT broker.
Timestamps
Time of each measure, for time series.
Device data
Identifiers and status of devices.
Reference data
Equipment and site data from your databases.
PROOF
IoT data at scale.
These references show sensor and IoT data processed with TIMi.
1.8 billion IoT data points per month
Codensa processes 1.8 billion IoT data points per month with TIMi.
More than 1,400 sensors
Volvo worked with data coming from more than 1,400 sensors.
FAQ
Common questions.
Which protocol is supported?
MQTT, with subscribe and publish actions. Other sources can be read from files, databases or web services.
Can it answer in real time?
Streaming flows answer in milliseconds for simple row-by-row processing. Steps that need all rows run in batch.
Do we need a dedicated cluster?
Typical IoT volumes run on one or two standard servers. Larger volumes can scale vertically first.
Can we use the data for predictive maintenance?
Yes. Prepared sensor data feeds Modeler for failure, downtime or degradation models. See the predictive maintenance use case.
RELATED USE CASES
Keep exploring.
Sensor Data Modeling at Scale
Model thousands of sensor variables together on a standard server.
Predictive Maintenance
Anticipate equipment failures from the data you already collect, with the reasons.
Data Integration
Extract, clean, load and verify data from 80+ sources on one server.
SEE FOR YOURSELF
Test it on your own data.
Run TIMi on your own data with a free trial, or talk to us about your case.
