Skip to main content
Journal of the Brazilian Computer Society
Download PDF

Recognizing and learning models of social exchange strategies for the regulation of social interactions in open agent societies

Journal of the Brazilian Computer Society volume 17, pages 143–161 (2011)Cite this article

Abstract

Regulation of social exchanges refers to controlling social exchanges between agents so that the balance of exchange values involved in the exchanges are continuously kept—as far as possible—near to equilibrium. Previous work modeled the social exchange regulation problem as a POMDP (Partially Observable Markov Decision Process), and defined the policyToBDIplans algorithm to extract BDI (Beliefs, Desires, Intentions) plans from POMDP models, so that the derived BDI plans can be applied to keep in equilibrium social exchanges performed by BDI agents. The aim of the present paper is to extend that BDI-POMDP agent model for self-regulation of social exchanges with a module, based on HMM (Hidden Markov Model), for recognizing and learning partner agents’ social exchange strategies, thus extending its applicability to open societies, where new partner agents can freely appear at any time. For the recognition problem, patterns of refusals of exchange proposals are analyzed, as such refusals are produced by the partner agents. For the learning problem, HMMs are used to capture probabilistic state transition and observation functions that model the social exchange strategy of the partner agent, in order to translate them into POMDP’s action-based state transition and observation functions. The paper formally addresses the problem of translating HMMs into POMDP models and vice versa, introducing the translation algorithms and some examples. A discussion on the results of simulations of strategy-based social exchanges is presented, together with an analysis about related work on social exchanges in multiagent systems.

References

  1. 1.

    Aaron E, Admoni H (2010) Action selection and task sequence learning for hybrid dynamical cognitive agents. Robot Auton Syst 58(9):1049–1056

    Article  Google Scholar 

  2. 2.

    Barbosa RM, Costa ACR (2010) Using CSP in the formal specification of the micro-organizational level of multiagent systems. In: Trappl R (ed) Cybernetics and systems 2010, Proceedings of the 20th European meeting on cybernetics and systems research, EMCRC 2010. Austrian Society for Cybernetic Studies, Vienna, pp 459–464

    Google Scholar 

  3. 3.

    Bernstein DS, Givan R, Immerman N, Zilberstein S (2002) The complexity of decentralized control of MDPs. Math Oper Res 27(4):819–840

    MathSciNet  Article  Google Scholar 

  4. 4.

    Blau P (1964) Exchange and power in social life. Wiley, New York

    Google Scholar 

  5. 5.

    Bordini RH, Hübner JF, Wooldrige M (2007) Programming multi-agent systems in AgentSpeak using Jason. Wiley series in agent technology. Wiley, Chichester

    Book  Google Scholar 

  6. 6.

    Carbonell JG (1980) Towards a process model of human personality traits. Artif Intell 15(1, 2):49–74

    Article  Google Scholar 

  7. 7.

    Castelfranchi C, Rosis F, Falcone R, Pizzutilo S (1997) A testbed for investigating personality-based multiagent cooperation. In: Proceedings of the symposium on logical approaches to agent modeling and design. Aix-en-Provence

    Google Scholar 

  8. 8.

    Castelfranchi C, Rosis F, Falcone R, Pizzutilo S (1998) Personality traits and social attitudes in multiagent cooperation. Appl Artif Intell 12:649–675

    Article  Google Scholar 

  9. 9.

    Cook K, Yamaguchi T (1990) Power relations in exchange networks. Am Sociol Rev 55:297–300

    Article  Google Scholar 

  10. 10.

    Costa ACR, Dimuro GP (2009) Introducing social groups and group exchanges in the PopOrg model. In: Proceedings of AAMAS 2009, vol 1. IFAAMAS, Budapest, pp 1297–1298

    Google Scholar 

  11. 11.

    Costa ACR, Dimuro GP (2009) A minimal dynamical organization model. In: Dignum V (ed) Handbook of research on multi-agent systems: semantics and dynamics of organizational models. IGI Global, Hershey, pp 419–445

    Chapter  Google Scholar 

  12. 12.

    Dimuro GP, Costa ACR (2006) Exchange values and self-regulation of exchanges in multi-agent systems: the provisory, centralized model. In: Brueckner S, Serugendo GDM, Hales D, Zambonelli F (eds) Engineering self-organising systems. LNCS, vol 3910. Springer, Berlin, pp 75–89

    Chapter  Google Scholar 

  13. 13.

    Dimuro GP, Costa ACR (2006) Interval-based Markov decision processes for regulating interactions between two agents in multi-agent systems. In: Dongarra J, Madsen K, Wasniewski J (eds) Applied parallel computing. LNCS, vol 3732. Springer, Berlin, pp 102–111

    Google Scholar 

  14. 14.

    Dimuro GP, Costa ACR, Gonçalves LV, Hübner A (2006) Regulating social exchanges between personality-based non-transparent agents. In: Gelbukh AF, Garça CAR (eds) MICAI 2006: Advances in artificial intelligence. LNCS, vol 4293. Springer, Berlin, pp 1105–1115

    Chapter  Google Scholar 

  15. 15.

    Dimuro GP, Costa ACR, Gonçalves LV, Hübner A (2007) Centralized regulation of social exchanges between personality-based agents. In: Coordination, organizations, institutions, and norms in agent systems II. LNCS, vol 4386. Springer, Berlin, pp 338–355

    Chapter  Google Scholar 

  16. 16.

    Dimuro GP, Costa ACR, Palazzo LAM (2005) Systems of exchange values as tools for multi-agent organizations. J Braz Comput Soc 11(1):31–50

    Article  Google Scholar 

  17. 17.

    Dimuro GP, Santos AV, Bedregal BC, Costa ACR (2009) Fuzzy evaluation of social exchanges between personality-based agents. In: Lopes LS, Lau N, Mariano P, Rocha LM (eds) New trends in artificial intelligence, Proceedings of the 14th Portuguese conference on artificial intelligence. APIA, Aveiro, pp 451–462

    Google Scholar 

  18. 18.

    Dur R, Roelfsema H (2010) Social exchange and common agency in organizations. J Socio-Econ 39(1):55–63

    Article  Google Scholar 

  19. 19.

    Emerson R (1976) Social exchange theory. Annu Rev Sociol 2:335–362

    Article  Google Scholar 

  20. 20.

    Franco MHI, Costa ACR, Coelho H (2010) Exchange values and social power supporting the choice of partners. Pueblos Fronteras Digital 6(9) (Available at http://www.pueblosyfronteras.unam.mx/, accessed in February, 2010)

  21. 21.

    Franco MHI, Costa ACR, Coelho H (2011) Simulating argumentation about exchange values in multi-agent interactions. In: Dimuro GP, Costa ACR, Sichman JS, Tedesco P, Adamatti DF, Balsa J, Antunes L (eds) Advances in social simulation, Post-proceedings of the Brazilian workshop on social simulation. IEEE, Los Alamitos, pp 105–112

    Google Scholar 

  22. 22.

    Ghasem-Aghaee N, Ören TI (2003) Towards fuzzy agents with dynamic personality for human behavior simulation. In: Proceedings of the 2003 summer computer simulation conference, Montreal, July 20–24, 2003. SCS, San Diego, pp 3–10

    Google Scholar 

  23. 23.

    Grimaldo F, Lozano M, Barber F (2007) Coordination and sociability for intelligent virtual agents. In: Sichman J, Noriega P, Padget J, Ossowski S (eds) Coordination, organizations, institutions, and norms in agent systems III. LNCS, vol 4870. Springer, Berlin, pp 58–70

    Chapter  Google Scholar 

  24. 24.

    Grimaldo F, Lozano M, Barber F (2007) A multiagent framework to animate socially intelligent agents. In: Innovations in hybrid intelligent systems. Advances in intelligent and soft computing, vol 44. Springer, Berlin, pp 17–24

    Chapter  Google Scholar 

  25. 25.

    Grimaldo F, Lozano M, Barber F (2007) Social animation in complex environments. In: Intelligent virtual agents. LNCS, vol 4722. Springer, Berlin, pp 379–380

    Chapter  Google Scholar 

  26. 26.

    Grimaldo F, Lozano M, Barber F (2008) Madem: a multi-modal decision making for social MAS. In: Proceedings of the 7th international joint conference on autonomous agents and multiagent systems, AAMAS’08, Estoril, May 2008. IFAAMAS, Richland, pp 183–190

    Google Scholar 

  27. 27.

    Grimaldo F, Lozano M, Barber F, Vigueras G (2008) Simulating socially intelligent agents in semantic virtual environments. Knowl Eng Rev 23(4):369–388

    Article  Google Scholar 

  28. 28.

    Hoare CAR (1985) Communicating sequential processes. Prentice-Hall, Englewood Cliffs

    Google Scholar 

  29. 29.

    Homans G (1961) Social behavior—its elementary forms. Harcourt, Brace & World, New York

    Google Scholar 

  30. 30.

    Kaelbling LP, Littman ML, Cassandra AR (1998) Planning and acting in partially observable stochastic domains. Artif Intell 101(1–2):99–134

    MathSciNet  Article  Google Scholar 

  31. 31.

    Kamphorst B, van Wissen A, Dignum V (2009) Incorporating BDI agents into human-agent decision making research. In: Aldewereld H, Dignum V, Picard G (eds) Engineering societies in the agents world X. LNCS, vol 5881. Springer, Berlin, pp 84–97

    Chapter  Google Scholar 

  32. 32.

    Ma E, Qu H (2011) Social exchanges as motivators of hotel employees’ organizational citizenship behavior: the proposition and application of a new three-dimensional framework. Int J Hosp Manag 30(3):680–688

    Article  Google Scholar 

  33. 33.

    Nair R, Tambe M (2005) Coordinating teams in uncertain environments: a hybrid BDI-POMDP approach. In: Bordini R, Dastani M, Dix J, Seghrouchni A (eds) Programming multi-agent systems. LNCS, vol 3346. Springer, Berlin, pp 1–37

    Chapter  Google Scholar 

  34. 34.

    Nair R, Tambe M (2005) Hybrid BDI-POMDP framework for multiagent teaming. J Artif Intell Res 23:367–420

    Google Scholar 

  35. 35.

    Oliehoek F, Visser A (2010) A decision-theoretic approach to collaboration: principal description methods and efficient heuristic approximations. In: Babuška R, Groen F (eds) Interactive collaborative information systems. Studies in computational intelligence, vol 281. Springer, Berlin, pp 87–124

    Chapter  Google Scholar 

  36. 36.

    Ören TI, Ghasem-Aghaee N (2003) Personality representation processable in fuzzy logic for human behavior simulation. In: Proceedings of the 2003 summer computer simulation conference, Montreal, July 20–24, 2003. SCS, San Diego, pp 11–18

    Google Scholar 

  37. 37.

    Paruchuri P, Bowring E, Nair R, Pearce J, Schurr N, Tambe M, Varakantham P (2006) Multiagent teamwork: hybrid approaches. In: Computer society of India communications. CSI (Invited Talk, available at http://teamcore.usc.edu/publications.htm)

  38. 38.

    Pereira D, Gonçalves LV, Dimuro GP, Costa ACR (2008) Towards the self-regulation of personality-based social exchange processes in multiagent systems. In: Zaverucha G, Costa AL (eds) Advances in artificial intelligence. LNAI, vol 5249. Springer, Berlin, pp 113–123

    Google Scholar 

  39. 39.

    Pereira DR, Gonçalves LV, Dimuro GP, Costa ACR (2008) Constructing BDI plans from optimal POMDP policies, with an application to agentspeak programming. In: Proceedings of conferencia Latinoamerica de informática, CLEI’08. SADIO, Santa Fe, pp 240–249

    Google Scholar 

  40. 40.

    Pereira DR, Gonçalves LV, Dimuro GP, Costa ACR (2011) An analysis of two regulation mechanisms for personality-based social exchange processes. In: Dimuro GP, Costa ACR, Sichman JS, Tedesco P, Adamatti DF, Balsa J, Antunes L (eds) Advances in social simulation, post-proceedings of the Brazilian workshop on social simulation. IEEE, Los Alamitos

    Google Scholar 

  41. 41.

    Piaget J (1995) Sociological studies. Routlege, London

    Google Scholar 

  42. 42.

    Rabiner LR (1989) A tutorial on hidden Markov models and selected applications in speech recognition. Proc IEEE 77(2):257–286

    Article  Google Scholar 

  43. 43.

    Rao AS, Georgeff MP (1991) Modeling rational agents within a BDI-architecture. In: Fikes R, Sandewall E (eds) Proceedings of the 2nd international conference on principles of knowledge representation and reasoning. Morgan Kaufmann, San Mateo, pp 473–484

    Google Scholar 

  44. 44.

    Rodrigues MR (2007) Social techniques for effective interactions in open cooperative systems. PhD thesis, University of Southampton, Southampton

  45. 45.

    Rodrigues MR, Costa ACR (2004) Using qualitative exchange values to improve the modelling of social interactions. In: Hales D, Edmonds B, Norling E, Rouchier J (eds) Proceedings of IV workshop on agent based simulations, MABS’03, Melbourne, 2003. LNAI, vol 2927. Springer, Berlin, pp 57–72

    Google Scholar 

  46. 46.

    Rodrigues MR, Costa ACR, Bordini R (2003) A system of exchange values to support social interactions in artificial societies. In: Proceedings of the 2nd international conference on autonomous agents and multiagents systems, AAMAS’03. ACM Press, Melbourne, pp 81–88

    Chapter  Google Scholar 

  47. 47.

    Rodrigues MR, Luck M (2006) Analysing partner selection through exchange values. In: Antunes L, Sichman J (eds) Proceedings of the VI workshop on agent based simulations, MABS’05, Utrecht, 2005. LNAI, vol 3891. Springer, Berlin, pp 24–40

    Google Scholar 

  48. 48.

    Rodrigues MR, Luck M (2006) Evaluating dynamic services in bioinformatics. In: Proceedings of the tenth international workshop on cooperative information agents, 2006, Edinburgo. LNAI, vol 4149. Springer, Berlin, pp 183–197

    Google Scholar 

  49. 49.

    Rodrigues MR, Luck M (2007) Cooperative interactions: an exchange values model. In: Noriega P, Vázquez-Salceda J, Boella G, Boissier O, Dignum V, Formara N, Matson E (eds) Coordination, organizations, institutions and norms in agent systems II. LNAI, vol 4386. Springer, Berlin, pp 356–371

    Chapter  Google Scholar 

  50. 50.

    Rodrigues MR, Luck M (2009) Effective multiagent interactions for open cooperative systems rich in services. In: Sierra C, Castelfranchi C, Decker KS, Sichman JS (eds) Proceedings of the 8th international joint conference on autonomous agents and multiagent systems, AAMAS’09, Budapest, May 2009. IFAAMAS, Richland, pp 1273–1274

    Google Scholar 

  51. 51.

    Schut M, Wooldridge M, Parsons S (2001) Reasoning about intentions in uncertain domains. In: Benferhat S, Besnard P (eds) Symbolic and quantitative approaches to reasoning with uncertainty. LNCS, vol 2143. Springer, Berlin, pp 84–95

    Chapter  Google Scholar 

  52. 52.

    Schut MC, Wooldridge M, Parsons S (2002) On partially observable MDPs and BDI models. In: d’Inverno M, Luck M, Fisher M, Preist C (eds) Foundations and applications of multi-agent systems, Selected papers from the UKMAS workshop. LNCS, vol 2403. Springer, Berlin, pp 243–260

    Chapter  Google Scholar 

  53. 53.

    Simari GI, Parsons S (2006) On the relationship between MDPs and the BDI architecture. In: Nakashima H, Wellman MP, Weiss G, Stone P (eds) Proceedings of the 5th international joint conference on autonomous agents and multiagent systems, Hakodate. ACM, New York, pp 1041–1048

    Chapter  Google Scholar 

  54. 54.

    Trigo P, Coelho H (2010) Decision making with hybrid models: the case of collective and individual motivations. Int J Reason-based Intell Syst 2(1):60–72

    Google Scholar 

  55. 55.

    Trigo P, Coelho H (2011) Simulating a multi-agent electricity market. In: Dimuro GP, Costa ACR, Sichman JS, Tedesco P, Adamatti DF, Balsa J, Antunes L (eds) Advances in social simulation, post-proceedings of the Brazilian workshop on social simulation. IEEE, Los Alamitos, pp 91–96

    Google Scholar 

  56. 56.

    Wooldridge M (2000) Reasoning about rational agents. Intelligent robots and autonomous agents. MIT Press, Cambridge

    Google Scholar 

  57. 57.

    Zafirovski M (2003) Some amendments to social exchange theory: a sociological perspective. Theory Sci 4(2) (Available at http://theoryandscience.icaap.org/, accessed in February, 2011)

Download references

Author information

Affiliations

  1. Centro de Ciências Computacionais, Universidade Federal do Rio Grande, Av. Itália km 08, Campus Carreiros, 96201-900, Rio Grande, Brazil

    Graçaliz Pereira Dimuro & Antônio Carlos da Rocha Costa

  2. Instituto Federal de Educação, Ciência e Tecnologia do Rio Grande do Sul, Núcleo Avançado de Farroupilha, Avenida São Vicente 785, 95180-000, Farroupilha, Brazil

    Luciano Vargas Gonçalves

  3. Programa de Pós-Graduação em Ciência da Computação, Universidade Federal do Rio Grande do Sul, Av. Bento Gonçalves 9500, 91501-970, Porto Alegre, Brazil

    Diego Rodrigues Pereira

Authors
  1. Graçaliz Pereira Dimuro
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Antônio Carlos da Rocha Costa
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Luciano Vargas Gonçalves
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Diego Rodrigues Pereira
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Antônio Carlos da Rocha Costa.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 2.0 International License ( https://creativecommons.org/licenses/by/2.0 ), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Reprints and Permissions

About this article

Cite this article

Dimuro, G.P., da Rocha Costa, A.C., Gonçalves, L.V. et al. Recognizing and learning models of social exchange strategies for the regulation of social interactions in open agent societies. J Braz Comput Soc 17, 143–161 (2011). https://doi.org/10.1007/s13173-011-0035-5

Download citation

Keywords

  • Social exchange strategy
  • Recognition and learning of social exchange strategies
  • Self-regulation of social exchange strategies
  • Partially observable Markov decision process
  • Hidden Markov model