Transcription
iGuard OEM SDKAPI Reference GuideiPrism Web Security800-782-3762www.edgewave.com
2014 EdgeWave Inc. All rights reserved. The EdgeWave logo, iPrism andiGuard are trademarks of EdgeWave Inc. All other trademarks and registered trademarks arehereby acknowledged.Other product and company names mentioned herein may be the trademarks of their respective owners.The iPrism software and its documentation are copyrighted materials. Law prohibits making unauthorizedcopies. No part of this software or documentation may be reproduced, transmitted, transcribed, stored in aretrieval system, or translated into another language without prior permission of EdgeWave,Inc.IG0001.1.1.0001iGuard OEM SDK API Reference Guide2
ContentsPurpose of this Document .4About the iGuard OEM SDK .4Thread Safety.4Architectural Considerations .4Initialization and Manager Lifecycle .4Threading/Process Model .5Categories.5Database and Memory Model .5Other.5Libraries.6Library Initialization and Shutdown .7Types .7Functions.7Ratings .8Accessing the iGuard Rating Library.8How to Get a Rating Database .10Ratings (Filters).10Rating an Object .12Setting and Testing Individual Ratings .15Error Reporting .17Appendix A: Sample Application .18Appendix B: Working with Ratings .23Technical Support.27iGuard OEM SDK API Reference Guide3
Purpose of this DocumentThis document presents the externally callable functions and type declarations necessary to employ theiGuard OEM SDK in your product. The intended audience for this document includes engineers skilledin the C programming language and familiar with web filtering concepts.It also assumes that you have a basic understanding and a working knowledge of the following:system and network securitysystem and network administrationthe Unix or Windows operating systemInternet protocolsAbout the iGuard OEM SDKThe iGuard OEM SDK, hereafter referred to as the SDK, contains all of the components required todevelop, compile and run SDK applications. The SDK contains the following directories:/include: This directory contains header files./lib: This directory contains libraries that are required to compile and run SDK applications./examples: This directory contains sample programs that demonstrate the use of the APIs.Thread SafetyThe loading of the filter list is not thread safe. In situations where this is a concern, we can makerecommendations on how to minimize this impact.Architectural ConsiderationsInitialization and Manager LifecycleWhen the Rating Manager is initialized, the filter database is read from disk. There are no diskaccesses after this point.When a new filter database is received from EdgeWave, the existing Rating Managershould be deleted and a new Rating Manager initialized, as index data within the filter may havechanged. No ratings lookups can be processed while the Rating Manager is being recreated.Setting up and tearing down a Rating Manager is expensive, so it is recommended that the RatingManager persist across connections and across large numbers of lookups.iGuard OEM SDK API Reference Guide4
Threading/Process ModelThe SDK is designed to be used by one thread in one process. If multiple clients are needed, wrapthe Manager-accessing thread with a gatekeeper thread/process that serializes requests to a singleRating Manager.CategoriesCategories are selected by string name, so if the category name set as supplied by EdgeWavechanges, a Rating Manager reload will be necessary.A Rating Manager reload cannot occur until a new filter database is available; once a newfilter database is available, both files are downloaded and the reload is initiated with the newfilter database.This will be announced to SDK customers weeks before it actually is released.This occurs rarely – historically no more than once every few years.The list of supported category names is deployed with the ratings database.Translation of categorySets to strings and strings to category names may be costly. As it involvesstring searching and parsing, categorySets (e.g., for whitelists and blacklists) should be createdrarely.Note: So that changes to the categories used to build categorySets will not require a recompile, EdgeWaverecommends using configuration files to store the names of categories thatcomprise categorySets (e.g., for whitelists and blacklists).Database and Memory ModelThe entire filter database is read into memory at initialization. The memory required depends onthe selection of categories that are available from EdgeWave.The full filter database (all possible categories) is 350MB, however, forplanning purposes you should plan for anticipated growth to a size of roughly 500MB.This memory remains allocated from the time that the Rating Manager is initialized until it isdeleted.OtherIt is recommended that no significant amount of URL caching be performed outside of the SDK. This isbecause the URL search strategy is log-base 2, so any cache deeper than 1 or 2 is likely less efficient thanthe main search unless there is a high cache hit percentage.iGuard OEM SDK API Reference Guide5
LibrariesThe following APIs are available in the SDK:Library Initialization and Shutdown: see page 7RTG ManagerRTG initRTG init memRTG init mem no ipRTG DeleteManagerCategories: see page 15RTG CategoryRTG FirstCategoryRTG NextCategoryRTG StringToCategoryRTG GetCategoryNameRatings and categorySets: see page 8RTG RatingRTG NewRatingRTG DeleteRatingRTG ClearRatingRTG LookupRTG LookupURLRTG LookupIPAddressRTG StringsToRatingRTG AddCategoryRTG DeleteCategoryRTG GetCategoryNamesRTG GetViolatingCategoriesRTG GetCommonCategoriesRTG ContainsCategoryRTG RatingToStringError Handling: see page 17RTG EnableTraceDebuggingRTG DisableTraceDebuggingiGuard OEM SDK API Reference Guide6
Library Initialization and ShutdownThe rating database must be loaded prior to calling any rating functions. The calling program should alsofree any resources allocated by the rating library before exiting. The following types and functions areused:TypesFunction NameDescriptionRTG ManagerThis opaque type represents an instance of a loaded rating database.RTG DeleteManagerFrees a previously allocated ratingManager.RTG INITThis enumeration type classifies the ways in which the ratingdatabase may be loaded. The defined values are:RTG init diskAll lookups are disk-based. This uses the minimal amount ofmemory. A cache is employed to speed lookups.RTG init memAll lookups are memory based. The initialize function will allocatememory for both the rating table, as well as an IP address map. Thiscan be substantial, but has the best speed performance.FunctionsRTG Manager *RTG NewManager(const char * ratingFile,RTG init type type);Create and initialize a new RTG Manager based on control file pathand type.voidRTG DeleteManager(RTG Manager ratingManager);Free a previously allocated ratingManager.iGuard OEM SDK API Reference Guide7
RatingsAccessing the iGuard Rating LibraryThe following types and functions are used to access the rating library.TypesFunction NameDescriptionRTG StringToCategory()Converts a category name to a RTG Category()Input:An existing manager, and the name of the categoryOutput:RTG NOCAT if name string does not exist in the DBThe proper RTG Category if it doesPreconditions:RTG Manager exists and is initializedAll strings in rating database are valid null-terminatedstrings.“name” is not nullPostconditions: noneRTG StringsToRating()The opposite of RTG GetCategoryNames(), generally used todefine a categorySetInput:An existing managerAn array of stringsA length of the array of stringsAn RTG Rating initialized with RTG NewRating()Output:An RTG Rating that fits the categories named if they arein the database; an empty RTG Rating if notAn integer with the number of categories setPreconditions:RTG Manager exists and is initialized;All strings in the rating database are valid null-terminatedstrings.names is a non-null array of non-null strings of lengthnamesLenRTG Rating “categorySet” has been allocated withiGuard OEM SDK API Reference Guide8
RTG NewRating()Postconditions:Contents of RTG Rating are modifiedFunctionsRTG CategoryRTG StringToCategory( RTG Manager *rtg ,intRTG StringsToRating( RTG Manager *rtg ,const char *name) ;const char **names ,int namesLen ,RTG Rating categorySet ) ;ExampleSee Appendix B: Working with Ratings for a sample of how to use the RTG StringToCategory() andRTG StringsToRating() functions to create three “classes” of ratings.iGuard OEM SDK API Reference Guide9
How to Get a Rating DatabaseRatings are available for download at the following site:http://filterupdates.stbernard.comNote: You will have your own path under this URL, and a username and password will be provided toaccess updates.Ratings (Filters)EdgeWave produces a new set of URL ratings (commonly known as a “filter” file) each day. The file isnamed YYMMDD000.rtg (YY last two digits of year, MM two-digit month number starting at 01,DD two digit day number starting at 01), and is available from your portal directory.Each of these filter files is complete and ready to be supplied to the SDK in an RTG NewManager call.Incremental FiltersYour account may be configured to make incremental filters available during the day. These filtersinclude new site ratings that are significant enough to be deployed before the next day’s filter update,such as malicious categories (e.g., Malware/Spyware).These incremental filters are produced only if new ratings of this type are identified, which maybe once a day or several times a day. They may be produced every hour, but no more frequentlythan that.These filters are downloaded from the same location (your portal directory) but have a differentnaming convention and usage.Once they are downloaded and used to create a revised .rtg file, they are ready to be supplied tothe filter library in an RTG NewManager call after destroying the existing manager (see“Initialization and Manager Lifecycle” on page 4).Incremental filters are “delta” files that take a full filter from one version to another. They arealways intra-day, and only advance one revision/version.The incremental filter file is named YYMMDDaaa-YYMMDDbbb.rtx, where YY, MM, and DDare as described above for Ratings (Filters). aaa and bbb are the “from” and “to” revisions. Thefirst version per day is always 000, so the first incremental file will be YYMMDD000YYMMDD001.rtx. The second incremental file in a day will be YYMMDD001-YYMMDD002.rtx,and so on. Important: If you want to use incremental files, you must apply them in order.The incremental filter file is an xdelta3 difference file. xdelta3 is a binary patching tool availableat http://xdelta.org/ (no endorsement implied). It is available for Windows, BSD, and Linux. Thistool is used to apply a incremental file (the .rtx file) to a full filter (.rtg) and produce a new fullfilter. This new filter is used in a RTG NewManager call. The new full filter is also used whenapplying the subsequent incremental filter.iGuard OEM SDK API Reference Guide10
ExampleTimeEventShell commandsAug 8, 20140900Full filter080808000.rtg isdownloaded fromthe portalAug 8, 20141100Incremental filter080808000080808001.rtx isdownloaded fromthe portalxdelta3 –d tgAug 8, 20141200Incremental filter080808001080808002.rtx isdownloaded fromthe portalxdelta3 –d tgSDK callsNotesRTG DeleteManager(),RTG NewManager("080808000.rtg")When a fullfilter isavailable, theSDK isrestarted withit.RTG DeleteManager(),RTG NewManager("080808001.rtg")The day's fullfilter is used toproduce the"next" fullfilterRTG DeleteManager(),RTG NewManager("080808002.rtg")The second fullfilter of the dayis used toproduce thethird filterRTG DeleteManager(),RTG NewManager("080809000.rtg")The previousday'sincrementalsand full filtersare discardedin favor oftoday's fullfilter(and so on untilthe next day.)Aug 9, 20140900Full filter080809000.rtg isdownloaded fromthe portalEtc. iGuard OEM SDK API Reference Guide11
Rating an ObjectThe rating interface presents a flexible API allowing the user to rate based on host, protocol, and path, oron a URL. The returned rating is an opaque type; further functions documented below are provided toaccess ratings by name.TypesFunction NameDescriptionRTG RatingThis opaque type represents a rating for an object. Accessorfunctions for ratings are described below.RTG GetCategoryCount()Counts the number of categories applicable to a given Rating orcategorySetInput:An RTG Rating, non-null, pointing to a valid RatingOutput:A count of the categories in the given RTG RatingPreconditions:RTG Rating “categorySet” must be allocatedwith RTG NewRating()Postconditions:nonePrototype:int RTG GetCategoryCount( RTG Rating rating ) ;FunctionsintRTG Lookup(const RTG Manager ratingManager,RTG Rating result,const char * proto,const char * host,const char * path);Lookup rating into result. proto must be one of “http”, “https”,“ftp”. host must point to full domain name (i.e. www.somesite.com) pathmay be null, in which case a domain only lookup will be performed,otherwise a best match on path will be performed. Returns boolean ifmatch was found.intRTG LookupURL(const RTG Manager ratingManager,RTG Rating result,iGuard OEM SDK API Reference Guide12
const char * url);This is the primary lookup function for SDK users. Performs the samerating lookup as RTG Lookup, but parses the URL into proto, host, andpath components first.iGuard OEM SDK API Reference Guide13
intRTG LookupIPAddress(const RTG Manager ratingManager,RTG Rating result,const char * ipAddress);Lookup rating(s) for ipAddress into result. ipAddress is in dotted-quadformat such “10.145.168.9”. Returns boolean if ipAddress was found indatabase.iGuard OEM SDK API Reference Guide14
Setting and Testing Individual RatingsThe RTG Rating opaque type represents a set of all possible ratings. These functions allow the user toset and test individual ratings within an RTG Rating, to get the human-readable name of a rating or allratings, and to test for the existence of subsets of ratings.RTG Rating RTG NewRating(void);void RTG DeleteRating(RTG Rating);void RTG ClearRating(RTG Rating);These functions respectively allocate a new RTG Rating instance, delete an RTG Ratingfreeing associated memory resources, and clear all ratings within an RTG Rating.RTG NewRating will return (RTG Rating) NULL if an allocation error occurs. NewRTG Rating objects are cleared on initialization.const char *RTG GetCategoryName(const RTG Manager ratingManager,RTG Category cat);Return an ascii string of the RTG Category name, i.e. “nudity”.intRTG GetCategoryNames(const RTG Manager ratingManager,RTG Rating rating,size t maxsize,char **result);Fill result with an ASCII representation for each category present in RTG Rating. Returns the numberof categories populated. maxsize indicates the size of result; this function will copy at most maxsizebytes of data to *result.intRTG RatingToString(const RTG Manager ratingManager,const RTG Rating rating,char * result,size t maxsize);Writes a string representation of the categories in rating to result, up to maxsize bytes. Returns 0for success.Note: this differs from RTG GetCategoryName in that it returns a string representation of all categoriesset in rating.RTG CategoryRTG FirstCategory(const RTG Manager ratingManager);RTG CategoryRTG NextCategory(const RTG Manager ratingManager,RTG Category previous);iGuard OEM SDK API Reference Guide15
Iterate over valid categories within ratingManager, return RTG NOCAT when all categories in theratingManager have been exhausted.intRTG AddCategory(RTG Rating rating,RTG Category category);Define category within rating. Returns 0 for success.intRTG DeleteCategory(RTG Rating rating,RTG Category category);Unset category within rating. Returns 0 for success.intRTG ContainsCategory(const RTG Rating rating,const RTG Category category);Return (boolean) if category is present within rating.intRTG GetCommonCategories(const RTG Rating rating,const RTG Rating policy,RTG Rating result);Create result that is the intersection of categories common to rating and policy. Return 0 if an erroroccurred, 0 if there are no ratings in the result set, 1 if there is one or more ratings in the result set.result must be valid, i.e. created and deleted by user.intRTG GetViolatingCategories(const RTG Rating rating,const RTG Rating onlyAllowed,RTG Rating result);Count the number of categories in rating that are not present in allowed – these are violatingcategories. result needs to be valid; i.e., created and deleted by user. Returns count 0 for success.iGuard OEM SDK API Reference Guide16
Error ReportingCurrently all errors (e.g., failed system calls, bad parameters passed to functions) are reported to stderr; itis recommended that these be redirected to logs. If tracing is turned on usingRTG EnableTraceDebugging causes “entry” and “exit” messages to be displayed for each API function.iGuard OEM SDK API Reference Guide17
Appendix A: Sample ApplicationThe main program parses the required command line arguments, initializes a rating manager, and callsProcessURLFile with the rating manager and the name of the URL file.ProcessURLFile opens the specified file with minimal error handling. A new rating is initialized via acall to RTG NewRating. The file is read line by line. As each line is read, line-end characters such ascarriage return ‘\r’ and newline ‘\n’ are eliminated, then a rating for the URL is o
The iPrism software and its documentation are copyrighted materials. Law prohibits making unauthorized . in the C programming language and familiar with web filtering concepts. . the filter database is read from disk. There are no disk
