eesti teaduste
akadeemia kirjastus
SINCE 1952
Proceeding cover
of the estonian academy of sciences
ISSN 1736-7530 (Electronic)
ISSN 1736-6046 (Print)
Impact Factor (2020): 1.045

Complex search: aggregation, discovery, and synthesis; pp. 89–106

Full article in PDF format | doi: 10.3176/proc.2012.2.02

Georg Singer, Dmitri Danilov, Ulrich Norbisrath


Search engines such as Google, Bing, and Yahoo, supplemented by other information search portals like Wikipedia, have become the means for searching information on the Internet. Along with the increasing popularity of search engines, the academic interest in search has shifted from analysing simple look-up query and response patterns to analysing rather complex information seeking needs. Current search tools seem to support complex search not as well as they do in the case of look-up. Especially support for aggregating search results from multiple search-queries, taking into account discoveries made during a complex search task, and synthesizing them to some newly compiled document of information is only at the beginning and motivates researchers to develop new tools for supporting those information seeking techniques. We focus in this work on the exploratory search concepts aggregation, discovery, and synthesis. Our idea is that these are today the most time consuming activities, especially when fulfilling a complex information need. We will use these three concepts throughout this paper to evaluate different approaches in exploratory search and give an overview of the state of the art and current ongoing research in respect to these concepts.


  1. Abraham, M. Motivation and Personality. Harper & Row, New York, 1954.

  2. Allan, J. HARD track overview in TREC 2005: high accuracy retrieval from documents. In Proceedings of the 14th Text Retrieval Conference. Massachusetts Univ. Amherst Center for Intelligent Information Retrieval, 2005.

  3. Amershi, S. and Morris, M. R. CoSearch: a system for co-located collaborative web search. In CHI ’08 Proceedings of the Twenty-sixth Annual SIGCHI Conference on Human Factors in Computing Systems. ACM Press, 2008, 1647–1656.

  4. Amigó, E., Gonzalo, J., Peinado, V., Peñas, A., and Verdejo, F. An empirical study of information synthesis tasks. In ACL ’04 Proceedings of the 42nd Annual Meeting on Association for Computational Linguistics. Association for Computational Linguistics, Barcelona, 2004, 207–214.

  5. Baeza-Yates, R. and Pino, J. A. A first step to formally evaluate collaborative work. In Proceedings of the International ACM SIGGROUP Conference on Supporting Group Work: The Integration Challenge. ACM New York, NY, USA, 1997, 56–60.

  6. Balinsky, H., Balinsky, A., and Simske, S. J. Automatic text summarization and small-world networks. In DocEng ’11 Proceedings of the 11th ACM Symposium on Document Engineering. ACM Press, 2011, 175.

  7. Bao, S., Xue, G., Wu, X., Yu, Y., Fei, B., and Su, Z. Optimizing web search using social annotations. In Proceedings of the 16th International Conference on World Wide Web. ACM New York, NY, USA, 2007, 501–510.

  8. Barzilay, R. and Elhadad, M. Using lexical chains for text summarization. In Proceedings of the ACL Workshop on Intelligent Scalable Text Summarization. 1997, Vol. 17, 10–17.

  9. Barzilay, R. and Elhadad, M. Inferring strategies for sentence ordering in multidocument news summarization. J. Artifl. Intell. Res., 2002, 17, 35–55.

10. Bates, M. The design of browsing and berrypicking techniques for the online search interface. Online Inform. Rev., 1989, 407–424.

11. Borlund, P. and Ingwersen, P. Measures of relative relevance and ranked half-life: performance indicators for interactive IR. In Proceedings of the 21st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. ACM New York, NY, USA, 1998, 324–331.

12. Broder, A. A taxonomy of web search. ACM SIGIR Forum, 2002, 36, 3.

13. Carmel, D., Zwerdling, N., Guy, I., Ofek-Koifman, S., Har’el, N., Ronen, I., Uziel, E., Yogev, S., and Chernov, S. Personalized social search based on the user’s social network. In Proceedings of the 18th ACM Conference of Information and Knowledge Management – CIKM ’09. ACM, Hong Kong, China, 2009, 1227–1236.

14. Church, K., Neumann, J., Cherubini, M., and Oliver, N. SocialSearchBrowser: a novel mobile search and information discovery tool. In Proceedings of the 15th International Conference on Intelligent User Interfaces. ACM, Hongkong, China, 2010, 101–110.

15. Cleverdon, C. W., Mills, J., and Keen, E. M. An Inquiry in Testing of Information Retrieval Systems. Aslib Cranfield Research Project, College of Aeronautics, Cranfield, UK, 1966.

16. Crimmins, F., Smeaton, A. F., Dkaki, T., and Mothe, J. TetraFusion: information discovery on the internet. In IEEE Intelligent Systems and Their Applications. 1999, Vol. 14, 55–62.

17. Ding, W. and Marchionini, G. A comparative study of web search service performance. In Proceedings of the ASIS Annual Meeting. 1996, Vol. 33, 136–142.

18. Dumais, S. T. and Belkin, N. J. TREC Experiment and Evaluation in Information Retrieval. Chapter “The TREC Interactive Track: Putting the User Into Search”. MIT Press, 2005.

19. Efe, K. and Ozerturk, S. Evaluating user effectiveness in exploratory search with TouchGraph Google Interface. In Human–Computer Interaction. New Trends. Springer, San Diego, 2009, 405–412.

20. El-Arini, K. and Guestrin, C. Beyond keyword search. In Proceedings of the 17th ACM SIGKDD International Conference of Knowledge Discovery and Data Mining – KDD ’11. San Diego, California, USA, 2011, 439–474.

21. Evans, B. M. and Chi, E. H. Towards a model of understanding social search. In Proceedings of the 2008 ACM Conference on Computer Supported Cooperative Work. ACM, San Diego, CA, USA, 2008, 485–494.

22. Freyne, J., Farzan, R., Brusilovsky, P., Smyth, B., and Coyle, M. Collecting community wisdom. In Proceedings of the 12th International Conference on Intelligent User Interfaces – IUI ’07. Honolulu, Hawaii, USA, 2007, 52–61.

23. Golovchinsky, G., Qvarfordt, P., and Pickens, J. Collaborative information seeking. Computer, 2009, 42, 47–51.

24. Gong, Y. and Liu, X. Generic text summarization using relevance measure and latent semantic analysis. In Proceedings of the 24th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. 2001, 19–25.

25. Grimes, C., Tang, D., and Russell, D. M. Query logs alone are not enough. In Workshop on Query Log Analysis at WWW. Citeseer, 2007.

26. Hachey, B., Murray, G., and Reitter, D. The embra system at duc 2005: query-oriented multi-document summarization with a very large latent semantic space. In Proceedings of the Document Understanding Conference (DUC) 2005, Vancouver, BC, Canada. 2005.

27. Haghighi, A. and Vanderwende, L. Exploring content models for multi-document summarization. In Proceedings of Human Language Technologies: The 2009 Annual Conference of the North American Chapter of the Association for Computational Linguistics. Association for Computational Linguistics, Stroudsburg, PA, USA, 2009, NAACL ’09, 362–370.

28. Heymann, P., Koutrika, G., and Garcia-Molina, H. Can social bookmarking improve web search? In Proceedings of the International Conference on Web Search and Web Data Mining. ACM Press, 2008, 195–206.

29. Hong, S. H., Nikolov, N. S., and Tarassov, A. A 2.5D hierarchical drawing of directed graphs. J. Graph Algorithms Appl., 2007, 11, 371–396.

30. Hong, L., Convertino, G., Suh, B., Chi, E. H., and Kairam, S. FeedWinnower: layering structures over collections of information streams. In CHI ’10 Proceedings of the 28th International Conference on Human Factors in Computing Systems. ACM Press, 2010, 947–950.

31. Huberman, B. A., Romero, D. M., and Wu, F. Social networks that matter: Twitter under the microscope. First Monday, 2009, 14, 8.

32. Jones, W. and Teevan, J. Personal Information Management (PIM)}. University of Washington, 2007.

33. Jones, R., Rey, B., Madani, O., and Greiner, W. Generating query substitutions. In Proceedings of the 15th International Conference on World Wide Web. ACM New York, NY, USA, 2006, 387–396.

34. Kelly, D., Dumais, S., and Pedersen, J. Evaluation challenges and directions for information seeking support systems. IEEE Computer, 2009, 42.

35. Kerne, A. and Smith, S. M. The information discovery framework. In Proceedings of the 2004 Conference on Designing Interactive Systems Processes, Practices, Methods, and Techniques – DIS ’04}. Cambridge, MA, USA, 2004, 357–360.

36. Koh, E., Kerne, A., and Hill, R. Creativity support: information discovery and exploratory search. In Proceedings of the 30th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval – SIGIR ’07. Amsterdam, The Netherlands, 2007, 895–896.

37. Krause, B. Hotho, A., and Stumme, G. A comparison of social bookmarking with traditional search. In Advances in Information Retrieval (Macdonald, C., Ounis, I., Plachouras, V., Ruthven, I., and White, R. W., eds). Springer, Berlin, Heidelberg, 2008, Vol. 4956, 101–113.

38. Kumar, N. and Benbasat, I. The effect of relationship encoding, task type, and complexity on information representation: an empirical evaluation of 2D and 3D line graphs. MIS Quarterly, 2004, 28, 255–281.

39. Lewandowski, D. The retrieval effectiveness of web search engines: considering results descriptions. J. Doc., 2008, 64, 915–937.

40. Lewandowski, D. Web Search Engine Research. Emerald Group Publishing, 2012.

41. Lewandowski, D. and Höchstötter, N. Web searching: a quality measurement perspective. Web Search, 2008, 14, 309–340.

42. Luhn, H. P. The automatic creation of literature abstracts. IBM J. Res. Dev., 1958, 2, 159–165.

43. MacCall, S. L. and Cleveland, A. D. A relevance-based quantitative measure for internet information retrieval evaluation. In Proceedings of the ASIS Annual Meeting. 1999, Vol. 36, 763–770.

44. Marchionini, G. Exploratory search: from finding to understanding. Commun. ACM, 2006, 49, 41–46.

45. Morris, M. R. A survey of collaborative web search practices. In Proceedings of the Twenty-sixth Annual SIGCHI Conference on Human Factors in Computing Systems. ACM, New York, NY, USA, 2008, CHI ’08, 1657–1660.

46. Morris, M. R. and Horvitz, E. SearchTogether: an interface for collaborative web search. In Proceedings of the 20th Annual ACM Symposium on User Interface Software and Technology. ACM, New York, NY, USA, 2007, UIST ’07, 3–12.

47. Morris, D., Ringel Morris, M., and Venolia, G. SearchBar: a search-centric web history for task resumption and information re-finding. In Proceedings of the Twenty-sixth Annual SIGCHI Conference on Human Factors in Computing Systems. ACM, New York, NY, USA, 2008, CHI ’08, 1207–1216.

48. Nenkova, A., Passonneau, R., and McKeown, K. The pyramid method: incorporating human content selection variation in summarization evaluation. ACM Transactions on Speech and Language Processing (TSLP), 2007, 4(2).

49. Nenkova, A., Maskey, S., and Liu, Y. Automatic summarization. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Tutorial Abstracts of ACL 2011. 2011.

50. O’Connor, B., Krieger, M., Ahn, D., Cohen, W., and Gosling, S. TweetMotif: exploratory search and topic summarization for twitter. In Proceedings of ICWSM. The AAAI Press, 2010, 2–3.

51. Ostrovsky, M. Information aggregation in dynamic markets with strategic traders. In Proceedings of the 10th ACM Conference on Electronic Commerce. ACM Press, 2009, 253–254.

52. Pickens, J., Golovchinsky, G., Shah, C., Qvarfordt, P., and Back, M. Algorithmic mediation for collaborative exploratory search. In SIGIR ’08 Proceedings of the 31st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. ACM Press, 2008, 315.

53. Qazvinian, V. and Radev, D. R. Scientific paper summarization using citation summary networks. In Proceedings of the 22nd International Conference on Computational Linguistics. Vol. 1. Association for Computational Linguistics, Stroudsburg, PA, USA, 2008, COLING ’08, 689–696.

54. Quirmbach, S. Universal search: Kontextuelle Einbindung von Ergebnissen unterschiedlicher Quellen und Auswirkungen auf das User Interface. In Handbuch Internet-Suchmaschinen (Lewandowski, D., ed.). Akademische Verlagsgesellschaft AKA, 2009, 220–248 (in German).

55. Robins, D. Interactive information retrieval: context and basic notions. Informing Science, 2000, 3, 57–62.

56. Schiffman, B., Nenkova, A., and McKeown, K. Experiments in multidocument summarization. In Proceedings of the Second International Conference on Human Language Technology Research. 2002, 52–58.

57. Schraefel, M. Building knowledge: What’s beyond keyword search? IEEE Computer, 2009, 42.

58. Silverstein, C., Marais, H., Henzinger, M., and Moricz, M. Analysis of a very large web search engine query log. In ACM SIGIR Forum. 1999, Vol. 33, 6–12.

59. Singer, G., Norbisrath, U., Vainikko, E., Kikkas, H., and Lewandowski, D. Search-logger analyzing exploratory search tasks. In Proceedings of the 2011 ACM Symposium on Applied Computing}. ACM, New York, NY, USA, 2011, SAC ’11, 751–756.

60. Smyth, B., Briggs, P., Coyle, M., and O’Mahony, M. Google Shared. A Case-study in Social Search. Springer. 2009, 283–294.

61. Steinberger, J. and Jezek, K. Using latent semantic analysis in text summarization and summary evaluation. In Proceedings of 7th International Conference ISIM ’04 Information Systems Implementation and Modelling. 2004, 93–100.

62. Taylor, B., Mayer, M., and Buyukkokten, O. Interface for a universal search engine, 2008. US Patent 7,447,678.

63. Tunkelang, D. Precision AND Recall. IEEE Computer, 2009, 42.

64. Twidale, M. B. and Nichols, D. M. Collaborative information retrieval. In Encyclopedia of Library and Information Sciences. Taylor & Francis, New York, 2010, Vol. 3, 1080–1087.

65. Vaughan, L. New measurements for search engine evaluation proposed and tested. Inform. Process. Manag., 2004, 40, 677–691.

66. Voorhees, E. M. and Harman, D. K. TREC: Experiment and Evaluation in Information Retrieval. MIT Press, 2005.

67. White, R. W. and Marchionini, G. Examining the effectiveness of real-time query expansion. Inform. Process. Manag., 2007, 43, 685–704.

68. White, R. and Roth, R. Exploratory Search. Morgan & Claypool Publishers, 2009.

69. White, R. W. and Roth, R. A. Exploratory search: beyond the query–response paradigm. Synthesis Lectures on Information Concepts, Retrieval, and Services, 2009, 1–98.

70. White, R. W., Muresan, G., and Marchionini, G. Report on ACM SIGIR 2006 workshop on evaluating exploratory search systems. In ACM SIGIR Forum. ACM New York, NY, USA, 2006, Vol. 40, 52–60.

71. Widjaja, Y. Y., Pang, C. N. I., Li, S. S., Wilkins, M. R., and Lambert, T. D. The interactorium: visualising proteins, complexes and interaction networks in a virtual 3D cell. PROTEOMICS, 2009, 9, 5309–5315.

72. Wilson, M., Russell, A., and Smith, D. A. mSpace: improving information access to multimedia domains with multimodal exploratory search. Commun. ACM, 2006, 49, 47–49.

73. Xu, S., Bao, S., Fei, B., Su, Z., and Yu, Y. Exploring folksonomy for personalized search. In Proceedings of the 31st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. ACM Press, 2008, 155–162

Back to Issue