A Inteligência Artificial é capaz de obter aprovação nos exames de suficiência e qualificação técnica do CFC?

Abstract

The relationship between Artificial Intelligence (AI) and accounting has become increasingly relevant in recent academic studies and in society as a whole, leading to reflections on its impact on the labor market, data security, transparency, and ethics in its use. This present study aims to analyze the performance of the ChatGPT, GPT-4, and Bard models in the exams conducted by the CFC (Federal Accounting Council) and in technical qualification exams such as QTG (General Technical Qualification), CVM (Securities and Exchange Commission), BCB (Central Bank of Brazil), SUSEP (Superintendence of Private Insurance), PREVIC (National Superintendence of Supplementary Pensions), and Forensic Accounting. We adopted a quantitative methodological approach to examine the performance of artificial intelligence (AI) in the proficiency and technical qualification exams administered by CFC in 2022. The GPT-4 model demonstrated remarkably superior performance compared to other models, achieving a 100% approval rate in proficiency exams and an 83% approval rate in technical exams in 2022, a result confirmed by the exact Fisher's test. Through homogeneity chi-square tests, applied to the frequencies of correct and incorrect responses in the selected exams, it was possible to evaluate the null hypothesis of consistent performance of an AI model across disciplines belonging to various knowledge areas. Regarding proficiency exams, the obtained results indicate a lack of significant evidence to reject the null hypothesis in the cases of ChatGPT and GPT-4. On the other hand, in the Bard model, the null hypothesis was rejected, indicating a more variable performance across disciplines. In technical exams, the results refute the null hypothesis, demonstrating that the AI models exhibited performance variations in the six analyzed disciplines. A presence of tables in the questions tends to impair the performance of AI models. However, only in GPT-4, a significantly superior performance was observed in questions without tables. Furthermore, the Tanimoto index revealed a moderate affinity among the responses of the three AI models. Finally, we investigated the models' ability to respond to open-ended questions in the QGT and CVM technical exams. It was observed that the ChatGPT and GPT-4 models demonstrated sufficient performance in the open-ended questions for approval in both exams, while the Bard model would only be approved in the QTG exam. Alignment with the Concentration Area (Control Management) and Research Line of the PPGCC (Controllership in Public and Private Entities): This dissertation advances the development of new knowledge and practices in the field of accounting by evaluating the performance of generative Artificial Intelligence (AI) models in acquiring professional certifications in the accounting sector. The integration of AI into controllership fosters advancements in knowledge and management control practices, opening a broad spectrum of opportunities for innovation and enhanced efficiency in financial data analysis, risk management, and the optimization of accounting processes, both nationally and internationally, applicable to both public and private entities. Furthermore, this research stimulates critical reflections on the formative needs of accounting students, considering the impact of AI on professional certifications. By addressing contemporary and emerging challenges at the intersection of AI and accounting, the study offers valuable insights for the future of the profession, redefining the essential competencies for accountants in the digital age. Consequently, this dissertation aligns with the PPGCC's objective of promoting systematic research on pertinent issues in Accounting Sciences, emphasizing the encouragement and creation of innovative tools, and contributing to the development of academic faculty in accounting.