Verification of a labor market domain using an academic crowdsourcing system

Full Text

Open Access Open Access
Restricted Access Access granted
Restricted Access Subscription or Fee Access


Students desiring to become a valuable good in the labor market are willing to pay a considerable monetary cost to obtain knowledge about their prospective job opportunities, nowadays with a diminishing interest in the obtainment of a diploma. Considering the behavior of the labor market as a domain theory under uncertainty, it is straightforward to expect the presence of contradictions, in the form of salaries unable to be classified due to high inconsistency and variation. We provide an algorithm to verify a labor market domain theory based on a crowdsourcing academic system, in which feedback about possible contradictions is generated as a result of consultations with experts inside of the market and clustered into different contexts. We found that the verification process can be repeated iteratively as long as the students’ overall tuition is equal or greater than a quantity partially defined by the number of different profiles of the students.

Full Text

Restricted Access

About the authors

Esteban J. Azofeifa

Peoples’ Friendship University of Russia (RUDN University)

Author for correspondence.

PhD student at the Department of Information Technologies

6, Miklukho-Maklaya St., Moscow, 117198, Russian Federation

Galina M. Novikova

Peoples’ Friendship University of Russia (RUDN University)


Candidate of Engineering Sciences, director of the Center of Development of Digital Technologies for Educational Processes and assistant professor at the Department of Information Technologies

6, Miklukho-Maklaya St., Moscow, 117198, Russian Federation


  1. M. G. Staton, “Improving Student Job Placement and Assessment Through the Use of Digital Marketing Certification Programs,” Marketing Education Review, vol. 26, pp. 20-24, 2016. doi: 10.1080/10528008. 2015.1091665.
  2. M. Scholten and N. Tieben, “Vocational qualification as safety-net? Education-to-work transitions of higher education dropouts in Germany,” Empirical Research in Vocational Education and Training, vol. 9, 2017. doi: 10.1186/s40461-017-0050-7.
  3. I. Fedorenko and P. Berthon, “Beyond the expected benefits: unpacking value co-creation in crowdsourcing business models,” AMS Review, vol. 7, pp. 183-194, 2017. doi: 10.1007/s13162-017-0106-7.
  4. J. Jiang, B. An, Y. Jiang, D. Lin, Z. Bu, J. Cao, and Z. Hao, “Understanding Crowdsourcing Systems from a Multiagent Perspective and Approach,” ACM Transactions on Autonomous and Adaptive Systems, vol. 13, 2018. doi: 10.1145/3226028.
  5. D. E. Difallah, M. Catasta, G. Demartini, P. G. Ipeirotis, and P. CudrMauroux, “The dynamics of micro-task crowdsourcing,” Proceedings of the 24th International Conference on World Wide Web - WWW’15, pp. 238-247, 2015. doi: 10.1145/2736277.2741685.
  6. E. J. Azofeifa and G. M. Novikova, “VUZ: a crowdsourced framework for scalable interdisciplinary curriculum design,” 2018 IV International Conference on Information Technologies in Engineering Education (Inforino), pp. 188-193, 2018. doi: 10.1109/inforino.2018.8581804.
  7. R. Baker, E. Bettinger, B. Jacob, and I. Marinescu, “The Effect of Labor Market Information on Community College Students’ Major Choice,” Economics of Education Review, vol. 65, pp. 18-30, 2018. doi: 10.1016/j.econedurev.2018.05.005.
  8. N. Engbom, “Firm and Worker Dynamics in an Aging Labor Market,” Federal Reserve Bank of Minneapolis, Working Papers 756, Apr. 2019. doi: 10.21034/wp.756.
  9. P. Pollok, D. Lüttgens, and F. T. Piller, “How Firms Develop Capabilities for Crowdsourcing to Increase Open Innovation Performance: The Interplay between Organizational Roles and Knowledge Processes,” Journal of Product Innovation Management, vol. 36, pp. 412-441, 2019. doi: 10.1111/jpim.12485.
  10. A. Sutcliffe, The Domain Theory: Patterns for Knowledge and Software Reuse. Mahwah NJ: Lawrence Erlbaum Associates, 2002. DOI: 10.1201/ b12455.
  11. H. S. Sugiarto, E. Lim, and N. L. Sim, “On analysing supply and demand in labor markets: Framework, model and system,” in Proceedings of the 6th IEEE International Conference on Data Science and Advanced Analytics, Research Collection School of Information Systems, 2019. doi: 10.1109/DSAA.2019.00066.
  12. A. Artikis, M. Sergot, J. Pitt, D. Busquets, and R. Riveret, “Specifying and Executing Open Multi-agent Systems,” in Social Coordination Frameworks for Social Technical Systems. Law, Governance and Technology Series. Cham: Springer, 2016, vol. 30. doi: 10.1007/978-3-31933570-4_10.
  13. T. Wiedemann, C. Manss, and D. Shutin, “Multi-agent exploration of spatial dynamical processes under sparsity constraints,” Autonomous Agents and Multi-Agent Systems, vol. 32, pp. 134-162, 2018. DOI: 10. 1007/s10458-017-9375-7.
  14. C. Fieseler, E. Bucher, and C. P. Hoffmann, “Unfairness by Design? The Perceived Fairness of Digital Labor on Crowdworking Platforms,” Journal of Business Ethics, vol. 156, pp. 987-1005, 2019. DOI: 10.1007/ s10551-017-3607-2.
  15. R. Casidy and W. Wymer, “A taxonomy of prestige-seeking university students: strategic insights for higher education,” Journal of Strategic Marketing, vol. 26, pp. 140-155, 2018. doi: 10.1080/0965254X.2016. 1182573.
  16. H. Buchs, E. Murphy, and M. Buchmann, “Landing a job, sinking a career? The trade-off between occupational downgrading and quick reemployment according to unemployed jobseekers’ career stage and job prospects,” Research in Social Stratification and Mobility, vol. 52, pp. 26-35, 2017. doi: 10.1016/j.rssm.2017.10.001.
  17. R. M. Smullyan, Theory of Formal Systems: Annals of Mathematics Studies. Princeton University Press, 1961.
  18. P. Cousot, “Verification by Abstract Interpretation,” in Verification: Theory and Practice. Lecture Notes in Computer Science. Berlin, Heidelberg: Springer, 2003, vol. 2772. doi: 10.1007/978-3-540-39910-0_11.
  19. G. M. Novikova, E. J. Azofeifa, and V. R. Milov, “Towards the formalization of contradictions in a legal multiagent system,” Procedia Computer Science, vol. 150, pp. 488-494, 2019. doi: 10.1016/j.procs.2019.02. 083.
  20. J. Eklundh and M. Björkman, “Recognition of Objects in the Real World from a Systems Perspective,” Kuenstliche Intelligenz, vol. 19, pp. 12-17, 2005.
  21. S. Shapiro, “Incompleteness and Inconsistency,” Mind, vol. 111, pp. 817- 832, 2002. doi: 10.1093/mind/111.444.817.
  22. P. Cousot, “Conflicts of Ontologies - Classification and Consensus-Based Methods for Resolving,” in Knowledge-Based Intelligent Information and Engineering Systems. KES 2006. Lecture Notes in Computer Science. Berlin, Heidelberg: Springer, 2006, vol. 4252. doi: 10.1007/11893004_ 34.
  23. D. H. Wexler, “Integrating computer technology: Blurring the roles of teachers, students, and experts,” in Computing and Educational Studies: A Special Issue of Educational Studies. Routledge, 2000, vol. 31, pp. 33- 43.
  24. C. M. Chen, C. Y. Liu, and M. H. Chang, “Personalized curriculum sequencing utilizing modified item response theory for web-based instruction,” Expert Systems with Applications, vol. 30, pp. 378-396, 2006. doi: 10.1016/j.eswa.2005.07.029.
  25. M. De Paola, V. Scoppa, and R. Nisticó, “Monetary Incentives and Student Achievement in a Depressed Labor Market: Results from a Randomized Experiment,” Journal of Human Capital, vol. 6, pp. 56-85, 2012. doi: 10.1086/664795.
  26. T. Collins, R. Hawkins, and E. M. Flowers, “Peer-Mediated Interventions: a Practical Guide to Utilizing Students as Change Agents,” Contemp School Psychol, vol. 22, pp. 213-219, 2018. doi: 10.1007/s40688-017- 0120-7.
  27. R. Suzuki, N. Salehi, M. S. Lam, J. C. Marroquin, and M. S. Bernstein, “Atelier: Repurposing Expert Crowdsourcing Tasks as MicroInternships,” in Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, New York, NY, USA: Association for Computing Machinery, 2016, pp. 2645-2656. doi: 10.1145/2858036. 2858121.
  28. A. Felce, S. Perks, and D. Roberts, “Work-based skills development: a context-engaged approach,” Higher Education, Skills and Work-Based Learning, vol. 6, pp. 261-276, 2016. doi: 10.1108/HESWBL-12-2015- 0058.
  29. J. Beck, K. Ostrow, and Y. Wang, “Students vs. skills: Partitioning variance explained in learner models,” in Student modeling from different aspects. Worcester Polytechnic Institute, 2016, pp. 2-9.
  30. G. M. Novikova, “Evaluation of the competence of personnel in the corporate management system [Otsenka kompetentnosti personala v korporativnoy sisteme upravleniya],” Competencies and expertise of the modern specialist [Kompetentsii i kompetentnost’ sovremennogo spetsialista], pp. 78-95, 2011, in Russian.
  31. Q. Zou, S. Xie, Z. Lin, M. Wu, and Y. Ju, “Finding the Best Classification Threshold in Imbalanced Classification,” Big Data Research, vol. 5, pp. 2- 8, 2016. doi: 10.1016/j.bdr.2015.12.001.
  32. S. Russell and P. Norvig, Artificial intelligence. Harlow: Pearson Education UK, 2016.
  33. L. A. Gottlieb, A. Kontorovich, and P. Nisnevitch, “Nearly Optimal Classification for Semimetrics,” Journal of Machine Learning Research, vol. 18, pp. 1233-1254, 2017. doi: 10.5555/3122009.3122046.

Copyright (c) 2020 Azofeifa E.J., Novikova G.M.

Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.

This website uses cookies

You consent to our cookies if you continue to use our website.

About Cookies