Benchmarking, validation and reproducibility of concurrent multiscale methods are still needed
Ellad B. Tadmor, Ronald Earle Miller · Modelling and Simulation in Materials Science and Engineering · 2017
At the time that we decided to write our 'fourteen methods' paper [1], concurrent multiscale approaches, like QC [2], CADD [3] and others, had reached a certain level of maturity.There were many groups that had techniques for coupling atomistic and continuum regions for the purpose of solving larger problems with atomic-scale accuracy where it was needed, at least for crystalline solids.Perhaps because it had reached this point, the zeitgeist of the day was 'benchmarking.'At conferences, the common sentiment was that as a community we needed a way to compare methods, as well as to agree on some benchmark problems that could be used for this purpose.It became clear, however, that choosing such a benchmark and performing a comparison would not be easy.Each group (ourselves included) developed methods that were in principle very general but were in practice manifested in codes that were specific.Some only worked with certain interatomic potentials, or for certain geometries, or some other limitation that made choosing a common benchmark problem very difficult.So we decided to do it ourselves.We were well positioned to implement as many methods of the day as we could in a common code.At the time we were working on a book on multiscale modeling [4] and we already had our own code that could run at least two of the methods (QC and CADD).We therefore felt that implementing the rest would be fairly straightforward.As it turns out-and in hindsight we realize this is no surprise-doing so was no easy task.Understanding a method in broad terms is not the same as understanding its details well enough to implement it.As we worked through those details it became clear that help would be needed.We contacted as many of the original authors as we could, and even visited some of them in person, to get the details sorted out.Two long years after we started this 'straightforward' task, we were able to make the comparison.We chose our benchmark problem, which itself was a nontrivial exercise (as explained in the article [1]), but our intention was to let the community develop this idea.By freely distributing the 'Multibench' code in which all fourteen methods were implemented [5], we hoped that others would extend the work, and study additional benchmark problems and new multiscale methods.This would test our conclusions and refine our understanding of best Modelling and Simulation in Materials Science and Engineering