WIXLIB

Interaction ExamplesLevelAggregation of objectsObject or packageIn-application renderingFile through filesystemFile as “raw” bitstreamSub-file data structureBitstream through I/OequipmentRaw signal stream through I/OequipmentBitstream on physical medium39Interaction ExamplesLevelAggregation of objectsObject or packageIn-application renderingFile through filesystemFile as “raw” bitstreamSub-file data structureBitstream through I/OequipmentRaw signal stream through I/OequipmentBitstream on physical medium4020

Interaction ExamplesLevelAggregation of objectsObject or packageIn-application renderingFile through filesystemFile as “raw” bitstreamSub-file data structureBitstream throughI/O equipmentRaw signal stream throughI/O equipmentBitstream on physical medium41Interaction ExamplesLevelAggregation of objectsObject or packageIn-application renderingFile through filesystemFile as “raw” bitstreamSub-file data structureBitstream through I/OequipmentRaw signal stream throughI/O equipmentBitstream on physical medium4221

Interaction ExamplesLevelAggregation of objectsObject or packageIn-application renderingFile through filesystemFile as “raw” bitstreamSub-file data structureBitstream through I/OequipmentRaw signal stream through I/OequipmentBitstream on physicalmediumVeeco Instruments. http://www.veeco.com/library/nanotheater detail.php?type application&id 78&app id 3443Multiple Paths for Viewing BitsCarrier, Brian D. "A Hypothesis-Based Approach to Digital Forensic Investigations." Doctoral44Dissertation, Purdue University, 2006. Figure 3-3 (p.60)22

Three Complicating Factors for Archivists:1. Medium Failure / Bit Rot2. Obsolescence3. Volatility45Bit Rot Preventing measures can help (proper storage andhandling), but bits on a given medium will eventual flip orbecome unreadableIn repositories We maintain integrity of bit stream through security,checksums, periodic sampling and other validation Bit rot and advantages of newer media both call forperiodic refreshing and reformattingBut: The media we receive may not be so well maintained Ensuring the integrity of the bit stream whentransferring from one medium to another is extremelyimportant4623

Obsolescence“Obsolete power corrupts obsoletely.”- Ted NelsonThe technology associated with interpreting therepresentation at each of the layers can changeor become less available47Order of Volatility Some types of data change much more quickly and oftenthan othersImportant to recognize in order to recover data from acomputer system or media, while ensuring that actionsdon’t make irreversible changes to their recordcharacteristicsExample: If the contents of the browser cache areimportant to you, capture the cache before using thebrowser4824

How and where does a computer storeinformation?49Bits – How Data are Conveyed inComputers Variable voltage electrical signals or pulses of lightBit represents a tiny “switch” with two possible states –on/off, true/false, 1/0Bit string or bitstream: a consecutive sequence of bits(e.g. 101000111010101)Rarely meaningful to humans – when looking atbitstream, usually use a hex editor (discussed later)5025

Motivations for Storage Hierarchy Different forms of memory/storage have significantlydifferent costs and performanceStore recent data close by, in fast, expensive, volatilestorageStore data that has not been used recently and is rarelyused in slower, cheaper, less volatile storage51Source: erarchy.svg5226

The Low-Level Building Blocks of Storage– Sectors and Clusters Your computer’s processor manipulates data in the formof bitstreams, and data is stored on your computer’shard drive as bitstreamsBut moving the data from the hard drive to the processordepends on higher-level chunks: sectors and clustersThink of mail sent to a member of a family who all live inthe same house – the envelope will indicate the houseaddress but won’t identify where that person’s bedroomis located within the house53Sectors Smallest unit of storage that can be assigned an address(i.e. can be directly identified & found by the computersystem)Have specified size, depending on the type of storage,e.g. CD-ROM 2048 bytes (2,352 including errorchecking) floppies (usually) 512 bytes modern hard drives 4,096 (previously 512 bytes)Created when disk is low-level formatted (usually bymanufacturer) with bad sectors identified by diskcontroller so data won’t be written to them5427

Clusters Groups of sectorsSmallest unit of storage that can be tracked by theoperating systemSizes depends on operating system, type & size ofstorage device – examples are 2048 bytes (4sectors of 512 bytes) or 4096 bytesDefined during high-level formatting performed byoperating system55Magnetic Disk (e.g. Hard Drive or Floppy) Bits stored as magnetic fields of different polarity Magnetized surface of disk rotates under a read/writehead Divided into tracks (like rings of atree) Tracks divided into sectors andclusters Windows: File Allocation Table(FAT) or Master File Table (forNTFS) indicates, for given file,what clusters contain its contentImage from : “Concepts.” In Active UNDELETE v2.0 Documentation. Active DataRecovery Software. www.active-undelete.com/3tracks.htm5628

Hard Drive Structure:A trackB sectorC sector of a trackD clusterSource: svg57Optical Media – CD-ROM as ExampleSource of Images: Compact Disk (CD). USByte.http://www.usbyte.com/common/compact disk 3.htm5829

Solid-State Drives html Uses integrated circuits to storedataNo moving partsCan be read using same I/Oequipment as used for hard drivesIncreasingly common in jpg-.html59Floppy DisksPhysical storage is similar to hard drivesdescribed above (magnetic charges in aspinning disk) Various types and sizes, e.g. high density,double density, 3.5 inch, 5.25 inch, 8 inch 3.5 inch floppies are relatively easy to readusing a USB drive, but older ones aremore complicated 6030

Floppy Controller HardwareCatWeasel1 (no longer available) Disc Ferret2FC 502541.2.3.4.5.6.Disk2FDI5Kryoflux3SuperCard BLE.htmhttp://www.cbmstuff.com/proddetail.php?prod SCP61Two Important Considerations for Internal Mediathat are Used as External Media Power - internal driveneeds different connector(often Molex), not the kindthat plugs into the wallCooling – when pulledfrom the computer, you’vealso separated the drivefrom the fan, so youshould often add anexternal one to ex female ns/SearchTools/item-details.asp?EdpNo 16485676231

Kryoflux Running on a “Mini JukeBox”**Adapted from a Mini JukeBox setup designed by the National Library of Australia63Areas Designed to Store Temporary Data Files on disk used for virtual memorymanagement – e.g. “swap files” in Windows95/98, “page files” in Windows NT/2000/XPTemp filesVarious caches - e.g. browser cache, whichincludes copies of recently downloaded files“Recent documents” in WindowsCookies – “expires” attribute can indicate quickdeletion or long-term retentionHistory files – e.g. browsing & download history6432

CachingStoring a copy of a subset of data from aslower data source to a faster (morereadily available) data source Examples: CPU cache from main memory Main memory cache from hard disk Hard disk cache from CD-ROM Proxy server cache from web sites 65Configuration and Log Files Often contain information about where files arelocated, when last opened, user preferences,state of files when last usedIn Windows, much of this happens in theRegistryOn a Mac, much of this happens in property list(p-list) filesAnother examples: Index.dat – RSS feeds, URLs visited, searchqueries and recently opened files in InternetExplorer6633

Windows Registry Information about: Applications installed Application settings Hardware installed Hardware settings User interface and system preferences User accounts Locations of files and recent activities, e.g. MostRecently Used (MRU) Lots of online activities, e.g. user names andpasswords, browsing and search query history67Representation Information6834

“No computation withoutrepresentation”Smith, Brian Cantwell. "Limits of Correctness inComputers." In Computerization and Controversy: ValueConflicts and Social Choices, edited by Rob Kling, 810-25.San Diego, CA: Academic Press, 1996. 815.69Rothenberg, Jeff. "Ensuring the Longevity of Digital Information." Washington, DC: Council on Library and Information Resources,1999.7035

Representation Information “Information that maps a Data Object into moremeaningful concepts” (OAIS) - makes humanlyperceptible properties happenExamples: file format, encoding scheme, datatypeReference Model for an Open Archival Information System (OAIS). CCSDS 650.0-M-2(Magenta Book). Consultative Committee for Space Data Systems, 2012. [ISO7114721:2012]. Figure 2-2.Reference Model for an Open Archival Information System (OAIS). CCSDS 650.0-M-2(Magenta Book). Consultative Committee for Space Data Systems, 2012. [ISO7214721:2012]. Figure 4-10.36

Reference Model for an Open Archival Information System (OAIS). CCSDS 650.0-M-2(Magenta Book). Consultative Committee for Space Data Systems, 2012. [ISO7314721:2012]. Figure 4-10.Representation Information can Reside inMany Places Within digital object itself Stored separately as metadata Encoded within software required to read andparse the digital object7437

Finding Representation InformationWithin a File Keys fields Headers Manifests75Rothenberg, Jeff. "Ensuring the Longevity of Digital Information." Washington, DC: Council on Library and Information Resources, 1999.7638

Not Just a Series of Bytes – Pointers andOffsets Pointer – reference within a file or programming code that leadsfrom one place to another Causes the data to be read out of serial order (i.e. a jumpfrom one place to another place that does not immediatelyfollow it within the data stream) Ability to resolve the pointer is essential Offset – location that’s some given distance from a starting point Location calculated by adding offset to a base address(location) Again, ability to resolve the offset to the precise locationwithin a bitstream is essential77Fonts and Character Encoding7839

FontPDF with missing fonts by prwheatly, on Flickr.79Font Determines how characters will appear on screen(generation of glyphs)Same character can appear completely different in twodifferent fontsCan be a major issue in digital preservation, whenconsistent rendering is importantNot usually a focus of digital forensics or data recoveryefforts, which focus on simply making sense of thecharacters within a bitstreamHowever, changes of fonts within a document canprovide some hints to versioning and authorship8040

Character Encoding81ASCII – Major “installed base” of Character Encoding,Designed for the English-Speaking WorldSymbol Decimal755856957:58;59 60 61 0000100000101000010010000118241

Unicode Huge number of possible characters – notlimited to 8-bits for each Mapped to unique codes (numbers) Standard first published in 1991 Current version is 8.0 (June 2015)83UTF-8 Unicode Transformation Format (UTF) set ofconventions for how specific Unicode code points arerepresented as unique byte sequences UTF 8 is widely used – including in email and web pages Codes 0 to 127 are backward compatible with ASCIISee: Frequently Asked Questions: UTF-8, UTF-16, UTF-32 & BOM. Unicode, Inc.http://www.unicode.org/faq/utf bom.html8442

Escape Codes and Character Entities When a system doesn’t allow use of certaincharacters (either because reserved for specialuses or because not allowed at all), must do atranslation to characters that it does allow Examples Inprogramming languages InHTML – use & or % convention InURLs - Use of “%” hexadecimal label85Special‐Use orDisallowed CharacterHexReplacementNotesSpace%20Extremely common when posting to Web from OS that allows white spaces infile names“%22#%23 %24%%25Imagine what problems this might cause!&%26Used within URL to separate query parameters %2B,%2C/%2F:%3AUsed as “anchor” within URLs (link to specific section of page)Used as separator between parts of a URL or directory path;%3B %3CCan appear when XML /HTML markup gets passed as part of URL %3DUsed within URL to assign parameter value %3ECan appear when XML /HTML markup gets passed as part of URL?%3FUsed within URL to indicate query parameters@%40Often appears as part of email address8643

Compression87Rothenberg, Jeff. "Ensuring the Longevity of Digital Information." Washington, DC: Council on Library and Information Resources,1999.8844

Three Levels of Compression*Format of file implements compressioninternally - e.g. body of JPEG file iscompressed but not file header Application creates completely new,compressed copy of file(s) – e.g. WinZip,gzip File system compresses data units – e.g.not writing data to series of sectors that areall filled with zeros *Carrier, Brian. File System Forensic Analysis. Boston, MA: Addison-Wesley, 2005.89Encryption9045

EncryptionSpecial data (“keys”) and algorithms usedto transform data into a form that ispurposely less easily readable Used for: Confidentiality Integrity Non-repudiation Authentication 91Encryption at Various Levels* Application that creates the fileApplication that reads an unencrypted file and creates anencrypted fileOperating System – “Before a file is written to disk, theOS encrypts the file and saves the cipher text to the dataunits. The non-content data, such as the file name andlast access time, are typically not encrypted. Theapplication that wrote the data does not know the file isencrypted on the disk.”Encrypt an entire volume – implemented in storagesystem below file system level*Carrier, Brian. File System Forensic Analysis. Boston, MA: Addison-Wesley, 2005.9246

Checksums – Compact Representationsof Bitstreams A given bitstream, fed into an algorithm, will generate a short stringof characters that is extremely unlikely to be generated by adifferent bistream fed into that same algorithmMost common MD5, SHA-1Can determine: If bits have changed after a transfer If bits have flipped within a storage environment Whether two different files are identical bitstreamsA library of hash values can identify “known and notable” (EnCaseterminology) files Known – files that can be ignored (e.g. software listed in NationalSoftware Reference Library) Notable – specific bitstreams that you’re trying to find93Checksums – Compact Representationsof Bitstreams Tools for checksum generation MD5Summer HashDeep MD5Deep Fileverifier FFMD5Drop or command-line tool (Mac) GtkHash (available in BitCurator environment) Many others 9447

In BitCurator environment: Right Click on File or Directory and Calculate MD5959648

97MD5 Hashes of an Entire Directory of Files9849

Hexadecimal Notation A more compact and more humanly readableway of representing a stream of bits Eachcharacter represents one of 16 possible values(0,1,2,3,4,5,6,7,8,9,A,B,C,D,E,F) Conveniently, a series of two characters representedin hexadecimal can represent exactly one byte (28 256 possible values) of data, because 162 256 Hex dumps from computer’s memory often usedfor debugging or reverse engineering softwareand for data recovery99How to Generate a Hex Dump Many free or inexpensive tools available for download, e.g.Cygnus Hex Editor, Hex Workshop, HexAssistant, HxD,Hex Fiend (Mac), GHex (Linux), MiniDumper*To generate your own hex dump from a given file, try:http://www.fileformat.info/tool/hexdump.htmHex viewing will usually include a separate view to the rightthat presents the ASCII equivalent of all bytes, which canhelp the human eye to detect patternsHex viewing only necessary when a file includes eithernon-ASCII strings of bits or corrupted file elementsIf file is composed completely of ASCII-encoded data,using a simple text editor (e.g. Notepad) is simpler way toview file contents* See http://en.wikipedia.org/wiki/Comparison of hex editors10050

Syllabus for a class (HTML File)Beginning of file tells uswhat kind of file it is101Slides from a lecture (PDF/A file)Contents of this PDF file are not as easy toread within ASCII view as the contents ofthe HTML file were, but note that, again,beginning of file tells us what kind of file it is10251

In the BitCurator environment:10310452

Let’s corrupt a bitstream.10510653

10710854

10911055

11111256

11311457

11511658

11711859

11912060

That doesn’t look right.Let’s compare it to our previous MD5 hash.12112261

Before and AfterFile NameChange MadeMD5 reak.pptOne haracter “C”(Hex 43)To:Character “D”(Hex 44)123Before and AfterFile NameChange MadeMD5 reak.pptOne haracter “C”(Hex 43)To:Character “D”(Hex 44)Note: A 1-byte change resulted in acompletely different MD5 hash of the file.12462

Now do it yourself!125Get yourself a Microsoft Word File:Create a n

Applying Digital Forensics to Digital Collections – Previous Work* Ross and Gow (1999) - potential relevanc e of advances in data recovery and digital forensics to collecting institutions More recently - active stream of literature related to use of forensic tools and methods for digital