A1 Journal article (refereed)
An experimental design for comparing interactive methods based on their desirable properties (2024)

Afsar, B., Silvennoinen, J., Ruiz, F., Ruiz, A. B., Misitano, G., & Miettinen, K. (2024). An experimental design for comparing interactive methods based on their desirable properties. Annals of Operations Research, Early online. https://doi.org/10.1007/s10479-024-05941-6

JYU authors or editors

Publication details

All authors or editorsAfsar, Bekir; Silvennoinen, Johanna; Ruiz, Francisco; Ruiz, Ana B.; Misitano, Giovanni; Miettinen, Kaisa

Journal or seriesAnnals of Operations Research



Publication year2024

Publication date17/04/2024

VolumeEarly online

PublisherSpringer Science+Business Media

Publication countryNetherlands

Publication languageEnglish


Publication open accessOpenly available

Publication channel open accessPartially open access channel

Publication is parallel published (JYX)https://jyx.jyu.fi/handle/123456789/94448


In multiobjective optimization problems, Pareto optimal solutions representing different tradeoffs cannot be ordered without incorporating preference information of a decision maker (DM). In interactive methods, the DM takes an active part in the solution process and provides preference information iteratively. Between iterations, the DM can learn how achievable the preferences are, learn about the tradeoffs, and adjust the preferences. Different interactive methods have been proposed in the literature, but the question of how to select the best-suited method for a problem to be solved remains partly open. We propose an experimental design for evaluating interactive methods according to several desirable properties related to the cognitive load experienced by the DM, the method’s ability to capture preferences and its responsiveness to changes in the preferences, the DM’s satisfaction in the overall solution process, and their confidence in the final solution. In the questionnaire designed, we connect each questionnaire item to be asked with a relevant research question characterizing these desirable properties of interactive methods. We also conduct a between-subjects experiment to compare three interactive methods and report interesting findings. In particular, we find out that trade-off-free methods may be more suitable for exploring the whole set of Pareto optimal solutions, while classification-based methods seem to work better for fine-tuning the preferences to find the final solution.

Keywordsoptimisationmulti-objective optimisationPareto efficiencydecision makingdecision-makersdecision support systemsinteractivityselection criteria

Free keywordsmultiple criteria optimization; interactive methods; performance comparison; empirical experiments; human decision makers

Contributing organizations

Related projects

Ministry reportingYes

Reporting Year2024

Preliminary JUFO rating1

Last updated on 2024-03-07 at 20:06