Transcription
International Journal of Computer Information Systems and Industrial Management Applications.ISSN 2150-7988 Volume 4 (2012) pp. 161-168 MIR Labs, www.mirlabs.net/ijcisim/index.htmlComputer Assisted Audit Tools and Techniques inReal World: CAATT's Applications and Approachesin ContextIsabel Pedrosa 1, Carlos J. Costa 21Instituto Superior de Contabilidade e Administração de Coimbra, ISCAC-IPCQuinta Agrícola, Bencanta, 3040-361, Coimbra, Portugalipedrosa@iscac.pt2Information Science and Technology Department, ADETTI-IUL, ISCTE-IULAv.ª das Forças Armadas, 1649-026 Lisboa, Portugalcarlos.costa@iscte.ptAbstract: Nowadays, Computer Aided Audit Tools (andTechniques’) support almost all audit processes concerning dataextraction and analysis. These tools were firstly aimed tosupport financial auditing processes. However, their scope isbeyond this, therefore, we present case studies and goodpractices in an academic context. Although in large auditingcompanies Audit Tools to do data extraction and analysis arevery common and applied in several contexts, we realized that isnot easy to find practical examples to teach this subject.Computer Science students’ and teachers usually use applicationmanuals’, which are adequate to explore the tools itself, but theyare not enough to do real data analysis: a “hands-on” approachis essential. Despite that, our students didn’t belong to anyComputer Science graduation. CAATTs are good examples ofsoftware aimed to help auditors to analyze data, to explore andstudy distinct data sets in auditing processes. CAATTs representan innovative approach to do data analysis, mainly because theyallow the manipulation of very large data records with nosignificant added costs. Frequently, users have anotherdifficulty: they are not able to adequate their data sets or toprepare their data in order to create useful input data, to dotheir analysis. In this paper, we present a proposal of extractionand data analysis, based on data sets obtained from a LMSplatform (Learning Management System), a real data set thatcan be used to support students learning about CAATTs topic.Fundamental procedures to reach specific answers using aCAATT as a main tool to do data extraction and analysis arealso referred. The context for this case study was an InformationTechnologies applied to Auditing course in a Master Degreegraduation.Keywords: CAATTs, Auditing, Information Systems, Frauddetection, E-Learning, Education, Knowledge discoveryinclude automated working papers and traditional wordprocessing applications” [2] or as “the use of certain softwarethat can be used by the auditor to perform audits and toachieve the goals of auditing” [3]. CAATTs can be classified“in four board categories: Data analysis software; Networksecurity evaluation software/utilities; Operating Systems andData Base Management System security evaluationsoftware/utilities; Software and code testing tools”, according[3].In this paper we are focused on Data Analysis softwaretools which, among other main uses, can be helpful to detectfraud evidences’: “evidence are gathered by interviews,document reviewing, CAAT use” [6].Several studies before state that mainly big companies havetools to do data analysis in data sets with large amount of datarecords [2], [7], [10]. Regardless of this usage and the fact thatthis tools are common in organizational context, students arenot very used to work with it in a learning context [16]:software full packages and licenses are expensive and is noteasy to use, create and prepared data sets. Sometimes, to takeadvantage of the empowerment of these tools, there is theneed to do a complex data “makeover” process to allow theuse of a data analysis tool, so the most time consumingoperation can be data understanding and data preparation,most similar to SPSS, a tool that Master students were alreadyfamiliar with. Other reason is software licenses: an educationversion could be enough to explore some data sets, howeversome vendors don’t have it or the process to achieve it iscomplicated and too much bureaucratic. However, the Returnon that Investment worth it: mainly, those efforts had goodresults among students.I. IntroductionNowadays, audit processes are supported by severalComputer Aided Audit Tools and Techniques, CAATTs.CAATTs can be defined as “any use of technology to assist inthe completion of an audit. This (broad) definition wouldII. ISCAC Contect about CAATTsISCAC, Instituto Superior de Contabilidade e Administraçãode Coimbra, a Polytechnic Portuguese Accounting,Administration and Management School, has a MasterDynamic Publishers, Inc., USA
162Graduation Degree on Audit for Public and OrganizationalEnterprises that includes a subject focused on InformationTechnologies Applied to Auditing (ICT on Auditing). Thefirst year was 2008/2009 and fifty-nine students attended thefirst edition; thirty-six, during the second and twenty duringthe third. Mostly, these students had their first contact withCAATTs in this subject. The assessment was based onstudents’ labor experiences, which included education andtraining, city council, audit organizations and accountingoffices. Students made use of that experience concerning theapplications they chose to do data analysis using CAATTssoftware. However, to introduce and to practice the CAATTs’contents, we realized we need data sets to test the concepts, allthe students at the same time, conducing to real learning andgood practices that they can use, later, in their specific datasets. A demo educational version was available for studentsand teachers: classrooms and students’ laptops were included.That educational version was limited to databases with lessthan 10.000 records. The full CAATT version “is adequatewhen data amount does not allow assuring audit processwithout a technological aid: paper amount will be impossibleto manage” [3]. The course was taught using a LearningManagement System, MOODLE, Modular Object-OrientedDynamic Learning Environment, supported by severalforums, documents, students’ contributions and submissions.As in [11] and [12], “the power of Internet in learningmanagement, trough an E-Learning platform, MOODLE, wasexplored as a tool to motivate students and to improve theirfinal skills”. In order to promote CAATTs’ usage, this projectwas also inspired in [13], where is stated that Audit CommandLanguage, ACL, an auditing software classified as DataAnalysis was used “to help students become more“technologically adept””.In many situations, original data sets are “prepared” justbefore being imported on CAATTs. A spreadsheet can beused on data “preparation”. However, using the CAATT, newdata fields can be added and calculated, taking other native ornon native fields – this is also an important moment on theprocess: it was considered a core operation to learn how toprogram using functions specific to this CAATT. TheCAATT functions’ syntax was not similar to any otherlanguage they learned before. Indeed, only few students thathad worked with DBase IV mentioned that the syntax waseasy to learn and to use and similar to Dbase. However, inorder to simplify this step, we’ve done the analogy withsimilar situations using a spreadsheet, a tool that all thestudents knew and felt comfortable with. Although formulassyntaxes’ are different, the arguments and the main aim weresimilar.III. Literature ReviewCAATTs can be defined as “any use of technology to assist inthe completion of an audit. This (broad) definition wouldinclude automated working papers and traditional wordprocessing applications” [2] or as “the use of certain softwarethat can be used by the auditor to perform audits and toachieve the goals of auditing” [3]. CAATs’ use is, nowadays,accepted in data analysis mainly because it is “useful whenlarge amounts of data are involved or complex relationshipsPedrosa and Costaof related data need to be reviewed programmatically to gleanappropriate evidence from the aggregated data” [4]. They also“increase the efficiency of the conclusions about dataanalysis” [5]. Despite all the arguments look like so recent, wecan conclude in [5], that CAATTs are not a new concept orapproach: this study “An Audit Approach to Computers: Anew practice manual” was published in 1978 and theparadigm is still changing as in [1]: GAS, Generalized AuditSoftware, running on Web 2.0, using a collaborative approachand several different profiles to improve Audit Softwareusage.CAATTs can be classified “in four board categories: ware/utilities; OS and DBMS security evaluationsoftware/utilities; Software and code testing tools”, according[3]. We can include IDEA, Interactive Data Extraction andAnalysis, and ACL, Audit Command Language, in the firstcategory and, among other main uses, can be tools used asfraud detection: “evidence are gathered by interviews,document reviewing and CAAT use” [6].According [7], auditors accepted the CAATs, but more recentstudies [8] and [9], suggest that CAATTs acceptance is verylow, it is variable according to the companies and it isdependent on the company dimension. In [9], the TechnologyAcceptance Model, TAM, was used to study InformationTechnologies' acceptance among auditors and “the resultsfrom an experiment with experienced auditors suggest thatfirms have the ability to influence the implementation of newtechnology by using longer-term budget and evaluationperiods and by communicating the approval of remotesuperiors regarding the software. In the absence of such firminterventions, the individual characteristics of the auditor(risk-aversion and perceptions of budgetary pressure)determine implementation decisions”.In [17], Unified Theory on Users Acceptance and Use onTechnology, UTAUT, studied the “examination of contextualfactors and individual characteristics affecting technologyimplementation decisions in auditing”. We realized that thecontact that auditors (present or in a near future) can have withthe CAATTs during a Master Degree, could be relevant tomitigate the individual characteristics, difficulties andresistances referred before. Consequently, we decided todefine specific supporting materials to teach Data AnalysesTools and to help teachers and students in this learningapproach. Other factors in CAATTs acceptance can berelevant, whoever, they will be future work.IV. Proposing a Process for data Analysis andExploration using a CAATTTo do an efficient Data Analysis, we can use the samemethodology defined as the main techniques on Data Miningand add new steps: Extraction Exploration, Transformation Model building and validation Deployment. Reuse.Extraction is the process to obtain the data set, usually fromdata center or a data server. On exploration andtransformation, the process “usually starts with datapreparation which may involve cleaning data, data
Computer Assisted Audit Tools and Techniques in Real Word: CAATT’s Applications and Approaches in Contexttransformations, selecting subsets of records and - in case ofdata sets with large numbers of variables ("fields") performing some preliminary feature selection operations tobring the number of variables to a manageable range” [18].Indeed, in this sentence the statement “to explore data” is usedwith the meaning of “data understanding as an elementarystep to be succeeded”. Databases, attributes and data types arenot easy to figure out and previous background knowledge onthe area can be a core preparation on accomplishing the rightresult [19]. This paper focus on exploration, transformationand on CAATTs’ usage to achieve efficient results.CAATTs to do Data Analysis may also be referred as a tool todeployment and to reuse: after we answered to core questionson the data, we can create macros to set some parameters andto repeat all the process later with similar but new data.V. Two Cases163Sample-Bank Transactions has the attributes described inTable 1, 1166 records and some of those records are in Figure1.Table 1. Bank Transactions Native sactionnumberCheque orDepositDay for thetransactionTransactionamountCommentsThis must be unique (primarykey)It must be filled inIt must be filled in with a datavalueDepending on transactionType, the amount must bepositive for Deposits (Credits)and negative for Cheques(Debits)Before we decided to create this Cases/Tutorials, we followedother approaches, namely, in the first year of the course,several IDEA data analyses applications were run duringclasses and assessments: IDEA for Education in “MOODLEdata analysis in a Polytechnic School” and “Students gradesand absences control in a Secondary School”; IDEA for auditorganizations: “Inventory control”; and IDEA for Accountingoffices: “Studding patterns using SAFT-PT file”. As theMOODLE data can become the most usable data (because ofthe number of schools in Portugal that is currently using thise-Learning platform, we choose to improve that case studyand to create a tutorial using data extraction from MOODLEServer at ISCAC, data transformation using IDEA and dataanalyses. The main aim was to study how could be possible toimprove MOODLE usage at ISCAC taking advantage ofstudents’ background as MOODLE users.A. CASE 1: using CAAT Data Bases as a first approach touse the tool1) The very beginners: using IDEA Data Bases to a firstapproach to CAATsIDEA has an example Project, composed by eleven databasesIDEA Metadata files, IMD extension, with data sets for a firstapproach to IDEA to beginners. Using that data set, our mainaims were: present, gradually, IDEA tool with examples to guide thestudents for real important data verification andvalidation, sums, patterns and error/”missings” detection; use native data sets (the ones distributed with thesoftware), easy to use, to learn basic functionalities onIDEA; to promote class/students interaction and knowledgediscovery based on students questions. The focus was onnew information and not on the tool.2) Elementary aims about CAAT toolTo achieve successfully our plan we choose Sample-BankTransactions: it has only four attributes, it was simple, all thestudents had background on this subject, and, as aconsequence on that, the focus was not on understandingcomplex data but on the right questions and processes wewanted the students to learn.Figure 1. Bank Transactions: first recordsUsing Sample-Bank Transactions, students were focused onthe answers to the following questions: Are there any null Deposits or null Cheques? If yes, inwhich dates did it happened? Are there other importanttransactions during the same reference days? Are there any duplicated transactions? Are there negative Deposits and/or positive Cheques?What are the dates? Are there other transactions duringthose days? Which are the transactions with the highest values? Canwe find a justification about it?Most of the previous questions were stated by students: after,the focused on learning how to answer the questions withIDEA.3) IDEA functionalitiesTo answer last questions, we realized that a “hands on”approach could the most effective. We just introduced brieflythe IDEA tool, explored the main aspects in the applicationmain window, as in Figure 2, and we started to answer thequestions stated just before.They learned how to: Use Field Statistics to find the null values and store it on anew database named null transactions: they found 4Deposits in 17-02-2008, 07-05-2008, 26-07-2008 and
164Pedrosa and Costa01-10-2008. Using the functionality Join Databases tocross null transactions with Sample-Bank Transactionsthey discovered all the transaction done concerning thereferred days. Use Duplicate Key Detection to detect the duplicatedtransactions: only one, the TransID 1505, referring to twoCheques -11.359,36 and -9.758,84. Use Summarization to group records by Type, create twonew databases one for Deposits (769 records) and theother to Cheques (397 records). Criteria and EquationEditor window were introduced to explain how to filterthe negative or positive amounts in each database. Noirregular records were found. Use Top Records Extraction to check the top 5extractions: a total of 7.826.192,90, mainly on 99 (TransID 420).After this exercise, planned for 2 hours, the students felt muchmore confident with this tool and the aims that we establishedjust before. We proceeded, analyzing the databasesSample-Detailed Sale and Sample-Sales Representatives. Toexplore the formulas and to use IDEA as a tool to discoverinformation in a specific and real context, we decided to useMOODLE data.IDEA analysis about MOODLE main aims’ is to exploreseveral IDEA functionalities using real data and a familiarapplication context for all the students, using IDEA as themain tool for data extraction and analysis. As in a data miningtool, we focused on data understand, to define the questionsconsidering the set we have, prepare and transform dataattributes to have a real answer. During all this steps wereviewed some aspects from our 3.1 example and have theopportunity to learn several new others. Adding macros to thistutorial in order to “redo” all this analysis with new dataextracted from MOODLE, was our last conquer.1) Data descriptionIn this study, we defined the following topics to do MOODLEusage analysis: courses created on MOODLE (courses table),Categories (courses cats table), forum usage (forum discusstable) and posts related to the forums created (forum poststable) and users registered on the platform (users table). TheEducational IDEA 8.04 had 10000 registers as limit for eachdatabase and this was the main reason why we didn’t get anyother indicators. However, we will refer that on topic“conclusions and future work”: indicators such as courses anduser activities and MOODLE platform logs are important datato know in a future work. For instance, we realized that thenumber of invalid logs on the system was very high and oneexplanation can be the new users’ creation process: the cancreate their own account and choose their password, even ifthey don’t belong to ISCAC. Sometimes, if they forgot thepassword, it is possible to create another user: the onlycriterion is that the email address can’t exist already inMOODLE users’ data base. However, this procedure canrepresent a lack of security and a risk to the organization. InTable 2, we have each database description with all theattributes and, in Figure 3, a COURSES CAT database view.The data set was extracted from version 1.9 .Table 2. MOODLE DatabasesFigure 2. Main IDEA Window ApplicationB. Case 2: LMS data analysis in a Polytechnic InstituteISCAC is a Portuguese Accounting, Administration andManagement School, focus on Accounting, Administration,Management, Informatics and Law (Solicitors). MOODLE isbeing used as an e-learning platform since September 2006.Considering the period from the beginning until August 2007,two scientific working papers were published about the usageof MOODLE among students and teachers at ISCAC. In [14]and [15] it is possible to find a previous data analysis, madewith Access and reporting to the first year of MOODLE usageat ISCAC, 2006-2007considering all the interactions betweenstudents and the platform, the contents available on courses asfora, chats, tests as interactive tools with course community,files and links. However, after this work no new studies aboutMOODLE courses, users and users-interaction with thisplatform were developed. The first time we use a similar dataset was on the Master Course 1st edition. After it, we realizethat this data set could join all the topics that we should focuson a course like this and, afterwards, this approach could be“exported” to other schools.NameContentsCOURSESAll MOODLE platformcourses, each with thisfields: ID (N), Category(N), Sortorder (N),Password (C),FullName(C),Shortname(C),Summary(C), Format(social, topics) (C),StartDate(N), Visible(N), Timecreated (N),Time
approach: this study “An Audit Approach to Computers: A new practice manual” was published in 1978 and the paradigm is still changing as in [1]: GAS, Generalized Audit Software, running on Web 2.0, using a collaborative approach and several different p
