In a world where manufacturing increasingly relies on data analysis to drive innovation, Spotfire is helping engineers across industries unlock insights that would otherwise remain hidden in complex datasets.
Brad Hopper, VP of Industry Applications at Spotfire, a business unit of Cloud Software Group, is well placed to explain how the company's visual data science platform is transforming high-tech manufacturing, particularly in the semiconductor industry, where data complexity and production demands continue to grow.
Founded in 1996, Spotfire emerged from the postdoctoral work of its founder, Christopher Ahlberg, whose research focused on creating better human interfaces for databases. The company's original mission – to use visual interaction to help humans find insights to recognise patterns, outliers and trends in large datasets – continues to drive its development today.
"The idea is bringing the human mind directly in contact with the data via interactive visual software," explains Brad. “We started our business focused on life science, specifically pharmaceutical research, and we were successful early on, supplying analytical software to biologists and chemists."
While Spotfire's technological capabilities have evolved significantly since those early days, its core purpose remains consistent.
The company has expanded from its initial focus on life sciences to serve multiple industries, including semiconductors, oil and gas and other advanced manufacturing sectors.
Helping manufacturers meet semiconductor demand
In the semiconductor industry, where manufacturing facilities can cost upwards of US$20bn, efficiency and effectiveness are paramount.
Clearly, substantial investments create immense pressure to accelerate production while maintaining quality.
"Making semiconductors is a very complex and expensive venture," says Brad. "If you make that sort of investment, you need to ensure that you are manufacturing in an effective and efficient way so you can – as rapidly as possible – get a return on that investment."
The technical challenges are equally daunting. Modern semiconductor manufacturing involves creating increasingly smaller components while packing billions of transistors into devices that often sell at relatively modest prices.
This creates what Brad Hopper describes as “immense time pressure on top of the technical pressure to design something and build it on a manufacturable process”.
Ultimately, achieving success requires an understanding of complicated data from multiple sources.
Visual data science: A powerful combination
At the heart of the Spotfire offering is what the company calls "visual data science", a combination of three key capabilities that work together to help engineers and scientists solve complex problems.
The first component is the visual exploration of data, allowing users to apply their expertise to investigate information and find answers.
The second is advanced analytics optimised for specific industries, enabling users to apply mathematical models to support visual analysis.
"The third piece is applications," Brad adds. "You can configure an analytic workflow that connects to operational systems, merges and transforms data, provides contextual views and applies sophisticated analytical routines to discover anomalies.
“Then, based on that result, the end user might drive in a different direction, investigating related information to find the root cause of that problem.”
While many software tools offer statistical calculations, machine learning capabilities or real-time monitoring, the Spotfire differentiator is how it brings together these elements to enhance human-computer interaction.
Brad continues: "We combine these capabilities in a unique way that lets engineers and scientists use their expertise, but maybe even more importantly, their creativity.
"They can explore, they can ask new questions and bring to bear statistical models, visualisation and real-time predictions. Spotfire helps people be at their most productive.”
Brad emphasises that Spotfire is designed for "smart people" – engineers and scientists with specialised knowledge who need tools that magnify their ability to ask and answer questions.
Limitations without visual data science
Manufacturers without access to visual data science platforms face significant challenges when dealing with the variety and complexity of data in modern manufacturing environments.
"None of that data is square – it’s all different shapes and sizes,” says Hopper. “You need to relate data across; you need to visualise the correlations; you need to calculate across those relationships; and then you often need to capture company-specific intellectual property and best practices for ways to interpret that data.”
“No other technology does this. Spotfire is still, as far as I can tell, unique in the market.”
The fragmentation of manufacturing technology ecosystems compounds the challenges faced by manufacturers without visual data science at their disposal.
Many organisations operate numerous software systems that simply don't communicate effectively with each other.
Brad adds: “What’s been missing in this market is a technology that's the engineer's best friend, helping you to connect all those different systems, merge the data and make calculations and ask and answer questions in the fastest possible way to solve problems."
In a world where manufacturing increasingly relies on data analysis to drive innovation, Spotfire is helping engineers across industries unlock insights that would otherwise remain hidden in complex datasets.
Brad Hopper, VP of Industry Applications at Spotfire, a business unit of Cloud Software Group, is well placed to explain how the company's visual data science platform is transforming high-tech manufacturing, particularly in the semiconductor industry, where data complexity and production demands continue to grow.
Founded in 1996, Spotfire emerged from the postdoctoral work of its founder, Christopher Ahlberg, whose research focused on creating better human interfaces for databases. The company's original mission – to use visual interaction to help humans find insights to recognise patterns, outliers and trends in large datasets – continues to drive its development today.
"The idea is bringing the human mind directly in contact with the data via interactive visual software," explains Brad. “We started our business focused on life science, specifically pharmaceutical research, and we were successful early on, supplying analytical software to biologists and chemists."
While Spotfire's technological capabilities have evolved significantly since those early days, its core purpose remains consistent.
The company has expanded from its initial focus on life sciences to serve multiple industries, including semiconductors, oil and gas and other advanced manufacturing sectors.
Helping manufacturers meet semiconductor demand
In the semiconductor industry, where manufacturing facilities can cost upwards of US$20bn, efficiency and effectiveness are paramount.
Clearly, substantial investments create immense pressure to accelerate production while maintaining quality.
"Making semiconductors is a very complex and expensive venture," says Brad. "If you make that sort of investment, you need to ensure that you are manufacturing in an effective and efficient way so you can – as rapidly as possible – get a return on that investment."
The technical challenges are equally daunting. Modern semiconductor manufacturing involves creating increasingly smaller components while packing billions of transistors into devices that often sell at relatively modest prices.
This creates what Brad Hopper describes as “immense time pressure on top of the technical pressure to design something and build it on a manufacturable process”.
Ultimately, achieving success requires an understanding of complicated data from multiple sources.
Visual data science: A powerful combination
At the heart of the Spotfire offering is what the company calls "visual data science", a combination of three key capabilities that work together to help engineers and scientists solve complex problems.
The first component is the visual exploration of data, allowing users to apply their expertise to investigate information and find answers.
The second is advanced analytics optimised for specific industries, enabling users to apply mathematical models to support visual analysis.
"The third piece is applications," Brad adds. "You can configure an analytic workflow that connects to operational systems, merges and transforms data, provides contextual views and applies sophisticated analytical routines to discover anomalies.
“Then, based on that result, the end user might drive in a different direction, investigating related information to find the root cause of that problem.”
While many software tools offer statistical calculations, machine learning capabilities or real-time monitoring, the Spotfire differentiator is how it brings together these elements to enhance human-computer interaction.
Brad continues: "We combine these capabilities in a unique way that lets engineers and scientists use their expertise, but maybe even more importantly, their creativity.
"They can explore, they can ask new questions and bring to bear statistical models, visualisation and real-time predictions. Spotfire helps people be at their most productive.”
Brad emphasises that Spotfire is designed for "smart people" – engineers and scientists with specialised knowledge who need tools that magnify their ability to ask and answer questions.
Limitations without visual data science
Manufacturers without access to visual data science platforms face significant challenges when dealing with the variety and complexity of data in modern manufacturing environments.
"None of that data is square – it’s all different shapes and sizes,” says Hopper. “You need to relate data across; you need to visualise the correlations; you need to calculate across those relationships; and then you often need to capture company-specific intellectual property and best practices for ways to interpret that data.”
“No other technology does this. Spotfire is still, as far as I can tell, unique in the market.”
The fragmentation of manufacturing technology ecosystems compounds the challenges faced by manufacturers without visual data science at their disposal.
Many organisations operate numerous software systems that simply don't communicate effectively with each other.
Brad adds: “What’s been missing in this market is a technology that's the engineer's best friend, helping you to connect all those different systems, merge the data and make calculations and ask and answer questions in the fastest possible way to solve problems."
Heterogeneous data and semiconductor innovation
The future of semiconductor innovation increasingly depends on manufacturers' ability to integrate and analyse heterogeneous data from multiple sources.
As manufacturers push the boundaries of what's physically possible, they need to harness every available data point.
"Our customers are pushing the envelope, pushing the limits of what's possible to make the tiniest line widths to pack billions and billions of transistors into devices," Brad notes. “To do that requires pulling out all the stops."
This means monitoring and analysing everything from process characteristics, to device performance and even facility conditions such as variations in ambient pressure or vibrations. Every data element has the potential to provide a critical edge.
Brad states: "They're trying to bring to bear every scrap of information and every bit of data from every possible source to get an edge over their competitors and to make that 20 billion factory effective and efficient so that they can get the return on their investment.”
Human and AI-powered exploration
Spotfire has embraced artificial intelligence throughout its development, from early statistical models to modern large language models (LLMs) and generative AI technologies.
One of the platform's core AI features is a recommendation engine that uses machine learning to identify interesting patterns in data automatically. This helps users find valuable starting points for their analysis.
"That ‘interestingness’ is part of how we use machine learning to surface anomalies in data," Brad goes on. “As an engineer or as a scientist, you might say, 'that looks interesting, I want to drill deeper into that’.
“You might mark a section of that distribution that looks unusual, and Spotfire will automatically identify correlations to that difference in behaviour."
More recently, Spotfire has integrated generative AI capabilities, allowing users to interact with the software through natural language. Users can ask questions about how to use the software, request specific visualisations or seek explanations of existing analyses.
The platform also supports integration with programming languages like Python and R, enabling users to create custom analytical functions – with or without coding expertise.
In the same way that upstream oil and gas engineers rely on subsurface models to make sense of complex geological data, manufacturing engineers often face similar challenges in visualising vast volumes of operational data across equipment.
Analogous to building a 3D geological model, Brad shares a recent moment that highlights the value of Gen AI in data-heavy environments: "Just the other day, I wanted to create a series of 3D surfaces from 16,000 pieces of operating equipment. I asked the copilot, ‘write a Python function for me that can grid and interpolate this information’.
“It wrote the program, injected it into Spotfire and all I had to do was point and click to say, ‘this is the data table I want to use; these are the values I want to display; go ahead and do your work’. Engineers and data scientists can work together to directly implement new functionality in the platform on their own or with AI assistance.”
Engineer personas and use cases
Spotfire serves several key engineering roles in semiconductor manufacturing, each with distinct needs and responsibilities.
Brad outlines three, beginning with product engineers, who define what devices will do, how they will function and under what conditions they will operate. They use Spotfire to analyse factory output data and define operating regions for different products.
Process and equipment engineers manage the manufacturing equipment and processes. They focus on ensuring equipment health, process specifications and optimal output for each manufacturing step.
Yield engineers work across the entire manufacturing spectrum to address performance issues and drive continuous improvement. They trace problems back to their sources and optimise factory operations to deliver better outcomes.
"Those three personas really make the world go around with respect to electronic and semiconductor manufacturing," highlights Brad. “And by the way, this is not just semiconductors, but also flat panel displays; it's printed circuit boards; it's componentry that goes into automobiles, aircraft and medical devices.
“This high-tech manufacturing – not just semiconductors, but also the surrounding ecosystem – is where the Spotfire technology is deployed.”
What next for Spotfire?
As Brad looks ahead, he reveals Spotfire is refocusing on its roots in domain-specific industries while developing more specialised capabilities for its core markets.
"Historically, our objective was always to provide the best-possible universal technology for looking at data," Brad explains. "But, customers are asking for more domain-specific visualisations, more application-specific calculations.”
This strategic shift includes investments in custom visualisations like wafer maps and bitmap visualisations that wouldn't be found in general visualisation software.
The company is also enhancing the underlying platform to enable further customisation by customers.
"We’re striving to become the high-tech manufacturing software of record for every company we can,” concludes Brad. “I know our customers will appreciate that because they've been pulling us to do it for so many years.
“We’re now investing in a very strong way in building out the domain-specific content that's so important for speaking the language of that given industry.”
Make sure you check out the latest industry news and insights at Healthcare Digital and be part of the conversation at our global conference series, Manufacturing LIVE.
Discover all our upcoming events and secure your tickets today.
Healthcare Digital is a BizClik brand
Company portals
Executives
Brad Hopper
VP of Vertical Markets at Cloud Software Group