104 blending formulas and flow assignments must be able to adapt. Figure 3 and Figure 4 together clearly demonstrate the capability of an MCTS solver to maintain consistent profits even as the feedstock composition varies dramatically through stockpiling and smart blending decisions. Even when the feedstock composition only varies slightly, and a set of different feedstock qualities are available (as in Scenario B), slight deviations in process conditions can make it difficult to anticipate product quality in advance. A stochastic, long-horizon approach such as MCTS is able to better account for these uncertainties in fulfilling client contracts while achieving higher profits at the same time, as demonstrated in Tables 6 and 7. The performance of greedy-guided MCTS relative to MCTS is informative in illustrating to what degree the greedy solution is a good prior versus a local optimum and potential trapping state. ggMCTS is underperforms MCTS in Scenario A because the greedy prior is actively detrimental, biasing the solver towards solutions that deprioritize long-term planning. However, since the feedstock varies less day-to-day in Scenario B, the greedy solution is a better starting point than nothing, so ggMCTS is almost always able to find a better solution than MCTS with the same computational budget. These results indicate that a greedy prior is useful in scenarios with low variability while it is better to start from scratch when there is a high degree of variability. It is worth noting that MCTS does not consistently perform better than the greedy benchmark, likely because MCTS struggles with large and continuous state and action spaces. The specific solver choice is not important, the intent is more to show that formulating the problem as an MDP makes the problem amenable to a wide range of stochastic RL approaches. From a purely solver-tuning perspective, the performance of MCTS could be significantly improved by discretizing the state and action space (which also more accurately reflects blending and routing in practice). Other approaches such as particle swarm optimization or training a deep neural network with MCTS (as in Kumar et al. 2021) could be considered as well. Although the two scenarios presented are synthetic cases, they demonstrate the potential of an optimization-under-uncertainty approach to improve feedstock blending and routing. The blending network model and MDP formulation are sufficiently versatile to represent almost any mining operation’s blending network. Additionally, one of the advantages of using a reinforcement learning approach like solving an MDP with MCTS is that its decision-making is interpretable. The flow assignments depicted in Figure 5 are easy to implement and make logical sense. 6. CONCLUSIONS In this work, we structured the problem of feedstock blending and routing as a Markov Decision Process over a blending network. Formulating the problem in this way allows for clear identification and formalization of the sources of uncertainty: ore and process variability. Then, stochastic optimization techniques such as Monte Carlo tree search (MCTS) can be employed to solve the blending problem while accounting for uncertainty for better performance. An MCTS solver was shown to achieve higher profits more consistently with increased client contract fulfillment (reflecting improved product quality) in two synthetic scenarios that were chosen to demonstrate how solvers interact with uncertainty over time and represent realistic situations that a mining operation may face. However, there are clear limitations to an MCTS solver, especially when the problem space becomes larger and more parameters are included to, for example, represent ore deposits and their variability more comprehensively or model process variability with greater granularity. Instead of seeking other RL solvers, we believe that the most fruitful next step would be to extend the MDP to a partially observable MDP (POMDP) formulation. In the current formulation, as an MDP, there is only transition uncertainty: solvers do not know exactly how the feedstock or process conditions will vary between today and tomorrow. There is no state uncertainty: at every point in time, the solvers know the composition of feedstock at every node. In reality, however, the blending network has significant state uncertainty. Without real-time chemical analysis, as is still often the case in the mining industry, feedback from chemical analysis is generally not immediate, and may be delayed by 1-2 days. Even with real-time chemical analysis, it is well-established in academia and in industry that process outputs are heavily dependent not just on the chemical composition of feedstock, but the texture and mineralogy as well, which are much more difficult and costly to measure. At the current state of technology, it would be impractical and
RkJQdWJsaXNoZXIy MTM0Mzk2