Design of Experiments has been part of precision manufacturing process development for decades. Response surface methods, full factorial and fractional factorial designs, central composite designs: the statistical foundation is solid, and DOE produces reliable results when applied correctly. The question we spent a lot of time on when building Reaxiomatic is not whether DOE works. It clearly does. The question is whether it is the right tool for high-mix precision machining, and in most cases we concluded it is not.
This article explains the difference between classical DOE and Bayesian parameter search, when each approach fits the problem, and where the practical tradeoffs live for a CNC shop with a diverse part mix.
What Classical DOE Assumes
Traditional DOE is a pre-planned experiment. You decide in advance which factors to vary, at what levels, and in what combinations. A two-level full factorial with five factors requires 32 runs minimum before you see a complete picture. A central composite design for surface response adds additional center points and star points. The number of required runs grows quickly with factors, and the run plan is fixed before the first chip is cut.
This structure works well when three conditions hold: the experiment can be conducted under controlled conditions without time pressure; the parameter space boundaries are well-understood; and the cost per run is low relative to the information gained.
In pharmaceutical process development, drug synthesis, or injection molding qualification, these conditions often hold reasonably well. The process runs in a lab or on a dedicated qualification machine. You can afford to run 32 or 64 combinations because the alternative is worse. The experimental design is generated once and run sequentially.
In a high-mix CNC machining environment, those conditions are almost never all true simultaneously.
Why the DOE Assumptions Break in High-Mix Shops
Consider a job shop running 15 to 30 distinct part numbers at any given time, taking on new contracts monthly, and running a mix of materials from 6061 aluminum to 17-4 stainless to Inconel 625. For each new part number, they need to establish working parameters: cutting speed, feed rate, depth of cut, coolant pressure, and tool path strategy. Each material and geometry combination is effectively a new experiment.
Running a proper DOE for each part number requires committing to a fixed run plan before production begins. That is expensive when the run plan may require 20 to 40 cuts before you get actionable results. More importantly, it requires that you know the relevant factor ranges before you start, and for a new material or geometry combination, you often do not. If you set factor levels too conservatively, you miss the optimal zone. If you set them too aggressively, you may violate tooling constraints or produce out-of-spec parts that cannot be sold.
There is also a sequencing problem. In a traditional DOE, the run order is typically randomized to control for time-related effects. In production, you cannot freely randomize runs because machine setups have cost. Deviating from the run plan to accommodate a hot order or a machine availability change partially invalidates the design.
The Bayesian Search Difference
Bayesian parameter optimization works differently at every step of the process. Instead of a pre-planned grid of combinations, it starts with a prior: an initial probabilistic model of how the outcome variable (yield, dimensional conformance, surface finish) responds to the input parameters. After each observation, the model updates. The next point to evaluate is chosen not by a fixed design, but by an acquisition function that balances exploration of uncertain regions against exploitation of currently high-yield zones.
The key practical difference is that the search adapts in real time. If the first three runs suggest that spindle speed above 4200 RPM consistently produces chatter on this workholding setup, the acquisition function shifts its attention away from that region. You do not have to run the remaining 29 combinations in a pre-planned grid that includes 10 more points in the chatter zone.
The convergence property of Bayesian optimization also means it finds workable settings in fewer total runs than a full factorial or response surface design covering the same parameter space. For a five-factor problem, a well-implemented Bayesian search typically converges on the high-yield zone in 15 to 25 observations, compared to 32 to 64 for the equivalent DOE. The savings compound across a part mix of 20 or 30 active jobs.
Where DOE Still Has the Advantage
Bayesian search is not strictly better than DOE in all situations. There are cases where the classical approach produces more reliable results.
When you need to characterize the full response surface, not just find the optimum, DOE is more informative. If you need to understand how yield degrades as you move away from optimal settings across multiple directions, a response surface design gives you a complete model. Bayesian optimization gives you the optimum and an uncertainty map around it, but it concentrates observations near high-value regions and may leave the low-yield regions undersampled if the acquisition function is tuned for pure exploitation.
DOE is also more appropriate when regulatory documentation requires a pre-planned experimental design. For certain medical device process validations and pharmaceutical manufacturing qualifications, the validation protocol requires a prospectively defined design. A Bayesian search, because it adapts mid-run, does not fit that structure. This is not a criticism of Bayesian optimization. It is a genuine constraint that process engineers in regulated industries need to account for.
What the Transition Looks Like in Practice
When we were building and refining our parameter search engine, the transition from thinking in DOE terms to thinking in Bayesian terms required a different mental model of what "an experiment" is. In DOE, you design an experiment and run it. In Bayesian search, you are running a series of observations that continuously update a model.
That means the early observations are information-gathering in a broader sense. The acquisition function during early iterations uses high uncertainty as a criterion for what to try next: it explores. As the model gains confidence about which regions of the parameter space are promising, it shifts toward exploitation: running points that are expected to be good based on the accumulated evidence.
The engineering team's role changes accordingly. With DOE, the engineer's judgment is front-loaded into the experimental design: which factors, what levels, what design structure. With Bayesian search, judgment is distributed across the process: reviewing the model's current uncertainty estimates, flagging physical constraints the model does not know about (tooling limits, fixture interference), and deciding when convergence is sufficient for production sign-off.
For high-mix shops where process engineers are spread across 20 or 30 active programs, distributing that judgment across the search process rather than concentrating it in a single upfront design step is a practical advantage. The search does not wait for an engineer to sit down and design the next experiment. It suggests the next point, the engineer validates it against physical constraints, and the run happens.
The Data Persistence Problem Both Approaches Share
One limitation applies equally to classical DOE and Bayesian search: the value of the observations is only as high as the quality of the data retention. A well-structured DOE run that produces detailed observation records is genuinely reusable: you can fit a new response surface model from it years later if the process conditions are similar. Bayesian search observations, because they are sequential and adaptive, are even more data-dependent.
In both cases, the observations need to be structured: parameter settings, outcome measurements, machine and tooling state, material batch information. Without that structure, the runs are just history. With it, they are a compound asset that gets more valuable as it accumulates.
This is one of the core problems we built Reaxiomatic around. Whether you are transitioning from DOE to Bayesian search or running either method today, the bottleneck is almost always not the optimization method. It is having the past run data in a form that the method can actually use.