WIXLIB

40283530Ratio to total (%)21252015Sample15Population10545222100Software dev. engineers (SDEs)NoviceSeniorLead SDEsNoviceArchitectsSeniorNovice0OtherSeniorJob titleAPIs the respondents reported learning includedone that provides access to personal informationmanager data on Windows mobile-based devices,classic windowing APIs, and Microsoft’s most recent Web application development platform.Survey ResultsI analyzed the data by identifying major categories of responses and labeling each response bycategory and subcategory. Because no clear trendemerged from the ranking of response data forboth strategies and obstacles, I ignored this facetof the survey.Responses on learning strategies yielded fewsurprises. Of the 80 respondents, 78 percent indicated they learned APIs by reading documentation, 55 percent used code examples, 34 percentexperimented with APIs, 30 percent read articles,and 29 percent asked colleagues. Lower-frequencyitems included a wide variety of other strategies,such as reading books or tracing code through adebugger. In addition to providing a general understanding of how developers approach APIs,7 thispart of the survey, along with the responses to thegeneral “comments” question, yielded few opportunities for deeper analysis. The rest of this articleis therefore concerned with the obstacles developers faced when learning APIs.I derived five major categories from the responses to the “obstacles” question (see Table 1).For each category, the table provides a descriptionand the number of associated respondents. A totalof 74 respondents mentioned at least one obstacle.I associated respondents with categories when theyindicated at least one obstacle in that category. Forexample, 50 (out of 74) respondents mentioned atleast one obstacle relating to API resources. Someresponses pertained to multiple categories.A major result of the survey is that resourcestopped the list of obstacles to learning APIs. Thisis a good reminder that efforts to improve the usability of an API’s structure2–4 need to be complemented by efforts to improve the resources available to learn them.The refined categorization of the resourcerelated responses elicited six categories with atleast eight associated respondents (listed under“Resources” in Table 1).Except for general gripes about the official documentation released with the API (“General”), thisclassification reveals the variety of challenges facing the personnel in charge of producing resourcesfor APIs. Namely, to mitigate obstacles, API documentation mustFigure 1. Distribution ofrespondents across jobtitles. Most respondents(49 percent) weredevelopers with junioror intermediate jobtitles. The distributionof respondents acrossjob titles closely mapsthat of the population. include good examples, be complete, support many complex usage scenarios, be conveniently organized, and include relevant design elements.A refined categorization of the structurerelated responses elicited two subcategorieswith at least eight associated respondents (listedNovember/December 2009 I E E E S O F T W A R E 29

Table 1Response categories for API learning obstaclesAssociatedrespondentsMain es caused by inadequate or absent resources for learning the API (for example, documentation)50ExamplesInsufficient or inadequate examples20GeneralUnspecified issues with the documentation14ContentA specific piece of content is missing or inadequately presented in thedocumentation (for example, information about all exceptions raised)12TaskNo reference on how to use the API to accomplish a specific task9FormatResources aren’t available in the desired format8DesignInsufficient or inadequate documentation on the high-level aspects ofthe API such as design or rationale8StructureObstacles related to the structure or design of the API36DesignIssues with the API’s structural design20Testing and debuggingIssues related to the API’s testing, debugging, and runtime behavior10BackgroundObstacles caused by the respondent’s background and prior experience17TechnicalenvironmentObstacles caused by the technical environment in which the API is used (for example, heterogeneous system,hardware)15ProcessObstacles related to process issues (for example, time, interruptions)13under “Structure” in Table 1). These subcategories confirm the generally held view that an API’sbasic design does impact its users. However, theresponses also bring to light that an API’s testability and the ease of reasoning about its runtime behavior also have an important impact. For example, a respondent indicated being hindered due toa “subtle difference in behavior of APIs dependingon context.”Hearing from RespondentsTo understand more deeply how developers dealwith new APIs in their daily work, I conducted aseries of 12 interviews with Microsoft softwaredevelopers recruited from survey respondents andpersonal contacts. The goal was to get a detailedpicture of important obstacles developers facedwhen learning new APIs, the context in whichthese obstacles occurred, and infer possible causesfor these obstacles. For this reason, I again chose anopen-ended, loosely structured style of qualitativeinterview,8 which consisted of asking participantsto summarize their work with the API and explainthe obstacles they faced. Interviews lasted between15 and 45 minutes and were audio-recorded.Although I conducted this study with Microsoft developers who might not be representativeof all API users, the results should be usable byothers working with APIs. Indeed, the main les30IEEE SOFT WAREw w w. c o m p u t e r. o rg /s o f t w a resons derived from the survey and interviews don’tconcern the frequency or predominance of specific trends, but a detailed interpretation of howdifferent situations played out in practice and thelessons we can derive from them.Emerging QuestionsFrom considering the aggregated survey results,reading the individual responses, and studying the interview transcripts, several importantthemes emerged. In choosing material for thisarticle, I favored themes that went beyond wellknown issues, as opposed to a systematic coverage of the concerns mentioned by respondents(survey) and participants (interviews). In doingso, I left out interactions that described valid butwell-known issues. For example, one participantdescribed the difficulty of choosing the rightfunction among alternatives in the Win32 API.The participant referred to the practice of addingfunctionality to an API without breaking backward compatibility by introducing “extensionmethods” with the “Ex” suffix:There is a function CreateWindow, and afunction CreateWindowEx. Ex creates somenew types of windows, which weren’t createdin the earlier functions of the API. So they aregrowing the [set of] functions, but sometimes

you have to read through the documentationto find out which function you should call.By now, API designers have explicitly recognized this practice as problematic.3In the end, the three themes that offered thebest insights were the necessity to understand theAPI’s design aspects and rationale on an as-neededbasis, obstacles related to using code examples,and the challenges of dealing with an API’s seemingly inexplicable behavior.Understanding Design Aspects and RationaleMany survey respondents expressed the feelingthat a lack of knowledge about the API’s high-leveldesign hindered their progress:I don’t understand the design intents behindthe API, the overall architecture, why certainfunctions are designed as such.But why do users need to know about an API’shigh-level design and the intent behind it? General API design guidelines include the principle of“low barrier to entry.” According to this principle,developers should design APIs so that their usersonly need to “take the time to fully understand thewhole architecture if they find a particular featureinteresting or if they need to move beyond simplescenarios.”3 But is the need to learn more of thedesign completely explained by the transition frombasic to advanced usage scenarios? As it turns out,moving beyond trivial usage involves many typesof decisions that can be informed by high-leveldesign. Seven participants specifically said that toproperly understand an API, they needed to understand its high-level design. The major insightresulting from this data is that knowledge of anAPI’s high-level design (and its rationale) can helpdevelopers choose among alternative ways to use it,structure their code accordingly, and employ it asefficiently as possible.One participant explained the issue directly:One of the things that I struggle with is tryingto figure out when there’re multiple ways ofdoing something. Which one is more appropriate? I’m not entirely sure Sometimes it’sdifficult to know without going and asking.But you don’t necessarily have time to go andask the experts, if you can find them.A different participant independently echoedthese comments on the value of experts for providing the API’s design context:I had the luck of actually working with themain guys that designed it, so I could go andask people: ‘Hey! Why is this like that?’ Andthey would give me the background.Choosing among alternative usages of an API isa decision that well illustrates the underlying challenge of understanding relevant parts of an API’shigh-level design on a need-to-know basis, as opposed to systematic study (as Janet Nykaza andher colleagues similarly observed7).Many participants also indicated, explicitly ortacitly, a desire to understand the design and rationale of the API to use it as efficiently as possible. Phrases such as “it would help you makeyour code better” and “use it the most efficiently”weren’t uncommon when participants spoke aboutthe value of design knowledge. One participant explicitly linked knowledge of design intent with asmooth API usage experience.Sevenparticipantssaid that toproperlyunderstandan API, theyneeded tounderstandits high-leveldesign.When you’re building a framework, there’san intent if you can understand what theintent was, you can often code efficiently,without much friction. If you don’t knowwhat the intent is, you fight the system.Working with Code ExamplesIn studies of developers, examples often emerge asa key learning resource.1,7,9 As Samuel McLellanand his colleagues summarize, “The code examplessupported several different learning activities, suchas understanding the purpose of the library, its usage protocols, and its usage context.”1 It’s no surprise that both survey respondents and interviewparticipants repeatedly mentioned code examples.In fact, more than one-quarter of all respondentsidentified the absence of API usage examples tailored to their needs as an obstacle to learning theAPI. My detailed analysis of the data largely confirmed McLellan and his colleagues’ observationsbut also explained in more detail how examplessupport API learning. In fact, studying how examples fail to support developers provided the richestinsights about the role of examples in API learning.We can divide code examples, very roughly,into three categories. In the first category (snippets), we find small code snippets intended todemonstrate how to access the basic API functionality. At Microsoft, technical writers authorsnippets provided in the Microsoft DeveloperNetwork Library (MSDN). A typical snippet onMSDN is the 30-line function showing how toread from and write to files. Tutorial examplesform the second category (tutorials). TutorialsNovember/December 2009 I E E E S O F T W A R E 31

Designdecisionsthat canimpact APIusage shouldbe traceablefrom thepoint-of-entrydocumentationpages.are typically longer, consist of multiple code segments, and form a more or less complete application. They can be embedded in prose and areintended to teach developers a specific aspect ofthe API. Tutorials can be found on MSDN aswell as in blogs and books. The third categoryof examples consists of code segments from complete applications (applications). Applications include both the demonstration samples sometimesdistributed with an API and open source projects that developers can download from varioussource code repositories.Examples can become more of a hindrancethan a resource when there’s a clear mismatch between the example’s purpose and the user’s goal.Most issues with examples were related to participants wanting to use snippets for purposes thatwent beyond basic interaction with the API. Specifically, participants referred to the following usages when discussing examples: providing “best practices” of an API’s use; informing the design of code that uses the API; providing rationale about an API’s design; and confirming developers’ hypotheses about howthings work.Among these, the most prevalent frustrationwas that snippets didn’t provide any support forthinking about “how to put things together:”The problem is always, when I feel I can’tmake progress when there’s multiple functions or methods or objects together that,individually they makes sense but sort of thewhole picture isn’t always clear, from just thedocs.In cases where snippets didn’t support the developer’s goal, there was sometimes a progression to the use of tutorials and applications. Forinstance, a participant explained how a scenarioinvolving multiple function calls wasn’t supportedby snippets in MSDN:So they have an example showing: “this ishow you create a connection, this is howyou issue a command with SQL.” It wasn’tclear from there what to do if I want to dotwo commands. [ ] So later I looked into abook and I learned that you have to close theconnection, you have to close the commandbefore you can issue another command. Iwas trying to issue two commands and I wasjust getting an exception.32IEEE SOFT WAREw w w. c o m p u t e r. o rg /s o f t w a reIn addition to the increased effort of usingbooks or downloading examples from the Internet, we can explain the progression from snippets to applications on an as-needed basis by othertrade-offs, including the need to validate examplesfound on the Internet.There are examples there [on the Internet] butsome of them seem to be quite old. [ ] Thoseexamples would still run but I don’t thinkthey have the most recent way of doing stuff.Unchecked obsolescence of available resourcesis only one reason why the collection of usercreated Internet resources (“the cloud”) isn’t theultimate solution for obtaining API learning resources. Another issue is the credibility of the examples’ source. Examples that are more stronglytied to the API’s creators seem more attractive because they seem to validate the rationale for usingthe API in a specific way:[The example] at least gets you thinking thatthey are doing it in this particular way sothere must be a reason why they chose thisparticular model [ ]. Even if I don’t completely understand why, I’ll do that anyway,just because that’s what they say the bestpractice is, and I assume that the people whodesign the framework have a pretty good ideaof what the best practice ought to be.However reasonable this assumption, it shouldbe made with care because the people who writeAPI documentation at Microsoft aren’t the peoplewho develop the API.Dealing with MagicRespondents often mentioned being puzzled byan API’s behavior and wanting to access its implementation to solve the puzzle. In his API designguidelines, Bloch encourages developers to “obeythe principle of least astonishment” because surprised users introduce bugs.5 Survey responsesshow that this general principle can be hard to putin practice. The interviews helped reveal specificfactors that astonish developers.Studying multiple cases of puzzling API behavior elicited a common trend: the API’s high-level design explained puzzling behavior to a large extentand wasn’t clearly reflected in the low-level documentation. Consequently, the API behaved as documented, but developers had difficulty discoveringand interpreting the explanation for its behavior.In one case, a participant using an activities

workflow API couldn’t understand why the different activities he created didn’t run in separatethreads. In this case, the API’s design modelwas counterintuitive, and the participant hadto read a book by the API’s developers to use itproperly.In another case, a participant who had learnedone API was working on a different API derivedfrom the first one. Incompatibilities between thetwo were unsettling:If you have a code base here [with one API]and you try to do the same thing [with theother API] and all of a sudden it doesn’twork, and you don’t know why. So, how doyou tackle that?For a third participant, one specific type ofuser interface component didn’t seem to detectclicks from mouse events, whereas all other similar components did. A nonstandard default valueexplained by the component’s role in the component hierarchy apparently caused this behavior.So, that’s an example of how you can dig itout of the docs post mortem, after you knowwhat happened, but you couldn’t predict thatbehavior ahead of time from the class-leveldocumentation.What these cases have in common are thatsome aspects of the API’s design have observableconsequences on the API’s behavior but aren’texplained clearly in its low-level documentation.As a result, they didn’t surface when developersfirst tried to use the API, leading to inexplicablebehavior.One coping strategy for such situations was adesire to inspect the API’s implementation. For example, one respondent indicated that an obstaclewasno view of the implementation. Good APIdesign shouldn’t require this, but in reality,understanding the internals can make a bigdifference in how well an API is used.This trend is interesting because it goes againstreceived knowledge about the principle of information hiding. As another example, in respondingto a question about what led him to look at theAPI source code, a participant answered,Binding, for example, has a lot of magic.A lot of “if your class is this class then wehave a special behavior for it, if it’s not, itdoesn’t.” These things are hinted at in thedocumentation, but it’s not clear what therules are. That’s where looking at the sourcecould help.In brief, a main observation from this analysis is that the reason for the puzzling behavior existed but wasn’t easily found because it related tohigh-level design concerns that weren’t referred toin point-of-entry API documentation. API documentation guidelines generally focus on the thoroughness of the low-level documentation,5 but acomplement to this guideline is that low-level documentation should address design decisions thatcan impact the API’s behavior.One overarching result of this study is thatthe resources available to learn an APIare important and that shortcomings inthis area hinder the API learning progress. Whenlearning APIs, developers interact with resourcesfor many purposes, such as discovering key information about the API’s high-level design. Developers in the study tried to understand part ofthe high-level design for many reasons, includingfinding out how to most efficiently use the APIand understanding subtle aspects of its behavior. Studying examples is an important strategyfor learning about design, but this approach ledto frustration when the examples weren’t welladapted to the task. Finally, some participantsperceived API behavior that seemed inexplicableat first to be a major obstacle.These observations have implications for APIusers, designers, documentation writers, and development tool builders. Developers stuck whilelearning an API should consciously try to matchtheir information needs with the type of resourcemost apt to provide the required knowledge. Forinstance, if the API’s behavior seems inexplicable,the answer might have as much to do with its design as with its low-level structure. Developerslooking for ways to interact with different APImethods along complex protocols might be morelikely to find good examples in advanced tutorialsand applications, rather than snippets.For API designers and documentation writers, these observations complement existing APIdocumentation guidelines by emphasizing thatdesign decisions that can impact API usage shouldbe traceable from the point-of-entry documentation pages and that user expectations about whatthey can get (and not get) out of a specific type ofNovember/December 2009 I E E E S O F T W A R E 33

About the AuthorMartin P. Robillard is an associate professor in the School of Computer Science atMcGill University. His research focuses on software evolution and maintenance. Robillardhas a PhD in computer science from the University of British Columbia. Contact him atmartin@cs.mcgill.ca.a welcoming environment, and the anonymousparticipants who made this research possible bygenerously sharing

way to interact with list structures). However, APIs have grown very large and diverse, which has prompted some to question their usability.1 It would be a pity if the difficulty of using APIs would nullify the productivity gains they offer. To ensure that this doesn’t happen, we need to know what ma