From Chemistry to Data Engineering: What Manufacturing Taught Me About the Value of Metrics

When these two align, everything runs smoothly. But when instincts clash with actual metrics, the consequences can be serious. In high-stakes environments, the difference between intuition and insight matters — sometimes more than we expect.

But let’s be honest: metrics aren’t perfect either. They evolve. Algorithms improve. Models adapt. That’s the beauty of working with data — it can always get better.

However, gut feelings don’t improve — it changes.

My Background in Chemistry Changed How I See Data

Before moving into data engineering, I trained as a chemist. And in that world, data isn’t just useful — it’s critical.

In chemical manufacturing, everything must be monitored constantly: temperature, pressure, humidity, flow rates. Missing a reading isn’t just a technical issue — it can result in serious safety incidents or environmental damage.

That mindset shaped how I approach data now. In chemistry, there’s no tolerance for assumptions. You either measure and monitor, or you risk failure.

IoT Sensors and Real-Time Monitoring: The Backbone of Modern Manufacturing

In most manufacturing environments today, metrics are captured using IoT sensors. Think of the basics — thermometers, pressure gauges, vibration sensors — all feeding data back to centralized systems in real time.

Most of the time, these sensors are connected to machines, providing early warnings of potential anomalies. This enables preventative maintenance, reduces downtime, and improves overall efficiency.

But collecting real-time data is just one piece of the puzzle. The real value is in storing and analyzing that data — even the anomalies and failures

Why Businesses Should Store All Operational Data — Including Mistakes

Storing raw sensor data, including data from failures, provides long-term value that’s often overlooked. Here’s why it matters:

1. Smarter Decision-Making

Operational issues tend to repeat. By logging previous failures and how they were resolved, companies can react faster and avoid costly delays.

2. Faster Error Detection

Understanding baseline patterns allows teams to identify when something is “off” much sooner — which translates into quicker maintenance and less downtime.

3. Improved Process Visibility

Patterns in historical data — such as recurring overheating after a certain number of hours — can help fine-tune operations and improve product consistency.

4. Higher-Quality AI Models

In manufacturing, you’re dealing with dozens (or hundreds) of parameters. Training machine learning models on real, internal data (rather than third-party sources) significantly reduces bias and improves accuracy.

5. Cost Efficiency

Storage is no longer a barrier. With cloud platforms like AWS, you can store gigabytes of operational data for a fraction of the cost — often as little as $0.023 per GB.

Coming from a background where data was a matter of safety, I carry that mindset into my work as a data engineer. Whether you're running a manufacturing plant or scaling a tech company, the principles are the same:

• Measure what matters
• Store more than you think you need
• Use data not just to report — but to learn

Because in the end, the real value of analytics isn't just knowing what happened. It's building systems that make better decisions possible.