DATA SCIENCE
R and Python integration: your code, inside the pipeline
R and Python integration with TIMi lets data scientists run their R, Python and JavaScript code as steps of Anatella pipelines, next to the compiled actions, with versioning of the code and a debugger for JavaScript.
R, Python, JavaScript
Write custom steps in the language you know.
Versioned code
Your scripts are versioned inside Anatella.
JavaScript debugger
Break points, watches and stack for JavaScript steps.
HOW IT WORKS
How R and Python integration works.
Three steps, in one engine, on a standard server with no cluster.
1. Prepare the data
Anatella loads and prepares the data at scale with its compiled actions.
2. Run your code
An R, Python or JavaScript step receives the data, applies your code and returns the result to the flow.
3. Reuse the step
The step becomes a box that colleagues can reuse in their own flows.
AnatellaAnatellaGETTING IT RIGHT
Choices that shape a mixed setup.
Use compiled actions for volume and your own code for what is specific.
Heavy lifting upstream
Your R or Python code runs at the speed of its own interpreter. Joins, aggregations and derivations over large volumes are best done by compiled actions first.
Version and document
Scripts are versioned inside Anatella. Name and describe each step so that colleagues can reuse it.
Know what is open
Steps coded in JavaScript, R or Python can be read and modified. Compiled actions are not open source.
TRANSPARENCY
Code you can read and review.
Custom steps are visible in the flow and their code is stored and versioned. A reviewer can read what a step does and compare versions.
For reviewers
Read the code behind each custom step.
For teams
Versions of each script are kept.
For governance
Your code runs on your own servers.
DATA
What data your code can work on.
Anatella connects to more than 80 sources, so this data can be gathered without prior custom development.
Prepared tables
Output of Anatella joins and aggregations.
Model scores
Scores computed with Modeler models.
Text fields
Text prepared by the text mining operators.
Sensor data
Aggregated sensor measures.
PROOF
Preparation at scale.
We do not have a published case study dedicated to R or Python integration. The reference below is indirect: it shows the volume of upstream preparation that precedes custom code.
5,000 wells, 500 sensors, 4,000 derived variables
Not documented as an R or Python case. It shows the volume of preparation Anatella handles before any custom modeling step.
FAQ
Common questions.
Which languages are supported?
JavaScript, R and Python. New operators can also be written in C++ with the SDK available to TIMi partners and clients.
Do I need R or Python to use TIMi?
No. The compiled actions and Modeler work without any code. R and Python are for teams who want custom steps.
Can I use my own libraries?
Your R and Python steps can use the libraries installed on your server, subject to your IT policies.
Is the speed the same as compiled actions?
No. Custom steps run at the speed of their interpreter. For large volumes, do the heavy transformations with compiled actions first.
RELATED USE CASES
Keep exploring.
SEE FOR YOURSELF
Test it on your own data.
Run TIMi on your own data with a free trial, or talk to us about your case.
