The Land Of Tcl/Tk - SQLite

Transcription

The Land Of Tcl/Tk

The Land Of Tcl/TkOther

Civilization

CivilizationTeaming Hordesof UnwashedHeathen Barbarians

The Land Of Tcl/TkBegin Here

The Land Of Tcl/Tk

The Land Of Tcl/Tk

The Land Of Tcl/Tk

The Land Of Tcl/Tk

The Land Of Tcl/Tk

ACID Most of SQL-92 Zero-configuration ServerlessUnlimited lengthBLOBs and CLOBsManifest typingSingle disk file Small footprint(225KiB) Cross-platformdatabases Fast Public DomainUser-definedfunctions

written in and for

SQLite Language Bindings Ada Fortran PHP Basic Haskell Pike C# Java Python Delphi Javascript Rebol Eiffel Lisp Ruby Euphoria Lua Scheme Flash Ocaml Smalltalk Forth Perl Tcl/Tk

SQLite Language Bindings AdaFortran PHP BasicHaskell Pike C#Java Python Delphi Javascript Rebol Eiffel Lisp Ruby Euphoria Lua Scheme Flash Ocaml Smalltalk Forth Perl Tcl/Tk T cl/T k is the only bindingincluded in the SQLitesource tree

Similar Design Philosophy Manifest Typing No arbitrary limits Designed for embedding “Just Works” Readable source code

Non-Comment Source LinesIn SQLite 3.3.7Tcl Bindings 8.55%Core C 35.71%Tcl 54.28%Other 1.46%

Non-Comment Source LinesIn SQLite 3.3.7Tcl Bindings 10100Core C 35.71%Other 1.46%

Uses For SQLite Replacement for clientserver RDBMSStand-in for enterpriseRDBMS during testingand demosLocal cache of enterpriseRDBMS data Persistence ofobjects, configurationoptions, etc.Complex datastructuresApplication file format

Uses For SQLite Replacement for clientserver RDBMSStand-in for enterpriseRDBMS during testingand demosLocal cache of enterpriseRDBMS dataTraditional Uses Persistence ofobjects, configurationoptions, etc.Complex datastructuresApplication file formatNon-traditional Uses

set fn [tk getOpenFile .]if { fn! ””} {set fd [open fn]# read and parseclose fd} File/Open reads and parses the entireapplication file Error prone Lots of code to maintain Application crash causes data loss

set fn [tk getSaveFile .]if { fn! ””} {set fd [open fn w]# Write the fileclose fd} Requires the user to remember to save Possible overwrite of independent changes Corruption if file generator and parser do notmatch

set fn [tk getOpenFile .]if { fn! ””} {sqlite3 db fn} No need to read and parse file contentUpdates are atomic, consistent, isolated, anddurableAutomatic concurrency control

Changes are written to disk immediately No data loss after unexpected power failure The Save and Save As options are obsolete –remove them.

Key ConceptSQLite wants to replacefopen() not Oracle

Key ConceptSQLite wants to replace[open] not oratcl

Go hereGet this

Or get this and doconfigure/make install

Can be omitted if SQLiteis installed on your system% source sqlite-3 3 8.kit% package require sqlite33.3.8% sqlite3 db database.db%N ew object forcontrolling the databaseN ame of the database file.A new one is created if it doesnot already exist .

Use the “eval” method to run SQLdb eval {CREATE TABLE users(userid INTEGER,first name VARCHAR(30),last name VARCHAR(40));}userfirst namelast nameSemicolon separates multipleSQL statements.Final semicolon is optional.

db eval {CREATE TABLE users(userid INTEGER,first name VARCHAR(30),last name VARCHAR(40));}userfirst namelast name

Fixed set of columnsuserfirst namedb eval {CREATE TABLE users(userid INTEGER,first name VARCHAR(30),last name VARCHAR(40));}last nameArbitrary number of rows

db eval {CREATE TABLE users(userid INTEGER,first name VARCHAR(30),last name VARCHAR(40));}userfirst namelast nameDatatypes are ignored, mostly

Use an I N SE RT statement to add datadb eval {INSERT INTO usersVALUES(1, 'D. Richard', 'Hipp')}

Better to specify column namesdb eval {INSERT INTO users(user,first name,last name)VALUES(1, 'D. Richard', 'Hipp')}

Columns can occur in any orderdb eval {INSERT INTO users(last name, user, first name)VALUES('Hipp', 1, 'D. Richard')}

db eval {INSERT INTO usersVALUES(1, 'D. Richard', 'Hipp')}userfirst name1D. Richardlast nameHipp

Use a SE LE CT statement to extractdata from the database“* ” means get all columns% db eval {SELECT *FROM users}1 {D. Richard} Hipp%

% db eval {SELECT *FROM users}1 {D. Richard} Hipp%Data returned in a T CL list

Better to specify specifc column namesrather than use “* ”% db eval {SELECT user, first name, last nameFROM users}1 {D. Richard} Hipp%

db eval {INSERT INTO usersVALUES(2, 'Ginger', 'Wyrick')}userfirst namelast name1D. RichardHipp2GingerWyrick

% db eval {SELECT * FROM users}1 {D. Richard} Hipp 2 Ginger Wyrick%Additional rows of data justmake the returned list longer

% db closeT hese do the same thing% rename db {}

% sqlite3 db database.db% db eval {SELECT * FROM user} {Script runs once forputs userid userideach row in resultputs “name first name last name”set}userid 1name D. Richard Hippuserid 2Column contents store inname Ginger WyrickT CL variables%

% sqlite3 db database.db% foreach {userid first name last name} \[db eval {SELECT * FROM user}] {puts userid useridputs “name first name last name”}userid 1name D. Richard Hippuserid 2name Ginger Wyrick%Similar to the previousexcept entire result setis held in memory here

% sqlite3 db database.db% db eval {SELECT * FROM user} {puts userid useridputs “name first name last name”break}userid 1“break” and “continue” workname D. Richard Hippin the usual way%

% db eval {SELECT * FROM user} break% set userid1% set first nameVariables persist afterD. Richardthe last iteration of theloop% set last nameHipp%

% db eval {SELECT * FROM user} var {puts userid var(userid)puts “name var(first name) var(last name)”}userid 1name D. Richard Hippuserid 2name Ginger Wyrick%

% db eval {SELECT * FROM user} var break% parray varvar(*) userid first name last namevar(first name) D. Richardvar(last name) Hippvar(userid) 1%List of result set column names

Use AS to specify alternative column namesin the result set% db eval {SELECT user AS id, first name AS fname,last name AS lnameFROM user} var break% parray varvar(*) id fname lnamevar(fname) D. Richardvar(lname) Hippvar(id) 1%

db eval {ALTER TABLE userADD COLUMN picture;}userfirst name1D. Richardlast namepictureHippNew Column Added

% set in [open drh.jpg]% fconfigure in -translation binary% set drhphoto [read in]% close in% db eval {UPDATE userSET picture drhphotoWHERE user 1}%N ote the use of curly-braces,not double-quotes

Go HereGet T his

% package require Tk% source sqlitecon.txt% sqlitecon::create .console {sqlite } test.db db%

Get email client source code here

proc sqlitecon:: edit {original text} {# Code here to implement a GUI editor# for original text and return the result.}db function edit ::sqlite:: editCreate a new SQL function named “edit ”implemented by the T CL proc “::sqlite:: edit ”

Go HereGet T his

The Land Of Tcl/Tk

Full Text Search

Full Text SearchQ: What is Full Text Search? A: In brief: what Google does.

Full Text Search Q: Why is full text search important? A: Internet search engines have users spoiled.Modern applications need to support full textsearch in order to be competitive

Full Text Search Q: Is it difficult to implement? A: It is tricky to make it efficient for large datasets and getting internationalization right is hard

Single Term Search

Multiple Term SearchT wo or moresearch termsper queryE ach resultcontains everysearch term

Restrict Search Terms“site:” keywordrestricts search toa single website

Phrase SearchT wo or more wordswithin double-quotesmust appear aswritten

OR SearchT wo or more wordsconnected by “OR”means only one isrequired per page

Excluding Terms“-” before a wordmeans show onlydocuments thatlack that word

Snippets“Snippets” areexcerpts of thedocument that matchthe search terms

Proximity search Some search engines allow queries like this:– sqlite NEAR postgresqlMatches when “sqlite” occurs near (within 10words of) “postgresql” in the documentNo more technically difficult than a phrasesearch

Elements Of Full Text Search Tokenizing Lexicon mapping Indexing and querying Snippet generation Scoring

h3 2006-Jun-19 - New Book About SQLite /h3 p a href "http://www.apress.com/book/bookDisplay.html?bID 10130" i The Definitive Guide to SQLite /i /a , a new book by a href "http://www.mikesclutter.com" Mike Owens /a .is now available from a href "http://www.apress.com" Apress /a .The books covers the latest SQLite internals as well asthe native C interface and bindings for PHP, Python,Perl, Ruby, Tcl, and Java. Recommended. /p hr width "50%"

h3 2006-Jun-19 - New Book About SQLite /h3 p a href "http://www.apress.com/book/bookDisplay.html?bID 10130" i The Definitive Guide to SQLite /i /a , a new book by a href "http://www.mikesclutter.com" Mike Owens /a .is now available from a href "http://www.apress.com" Apress /a .The books covers the latest SQLite internals as well asthe native C interface and bindings for PHP, Python,Perl, Ruby, Tcl, and Java. Recommended. /p hr width "50%"

h3 2006-Jun-19 - New Book About SQLite /h3 p a href "http://www.apress.com/book/bookDisplay.html?bID 10130" i The Definitive Guide to SQLite /i /a , a new book by a href "http://www.mikesclutter.com" Mike Owens /a .is now available from a href "http://www.apress.com" Apress /a .The books covers the latest SQLite internals as well asthe native C interface and bindings for PHP, Python,Perl, Ruby, Tcl, and Java. Recommended. /p hr width "50%"

p a href "http://www.sqlite.org/index.html" img style "border: 0px none ; margin: 5px;" src "bibliophile.new files/sqlite.gif" /a /p p SQLite �在 NB 上寫 Rails ,但是又不想裝 MySQL 好重,第一個就想到 SQLite 。關於 SQLite 的中文介紹,可以參考 a href "http://www.dev.idv.tw:8080/." 簡介 SQLite /a  和  a href "http://openvanilla.org/wiki/zh/index.php" SQLite 使用教學 /a 。基本上  sqlite 就是一隻 command-line 程式而已,再加上一個資料檔( 即一個資料庫 ) 。 /p p 要在 Rails 中使用,首先去 a href "http://www.sqlite.org/index.html" SQLite /a 網站下載 執行檔 跟 DLL 檔,即 sqlite3.exe 跟 sqlite3.dll ,放到 C:/ruby/bin 下。 /p p 接著安裝 sqlite for Ruby gem ,執行 gem install sqlite3-ruby ,並選擇 win32的版本。 /p p 最後設定 database.yml : /p

2006 Jun 19 New Book About SQLite The Definitive Guideto SQLite a new book by Mike Owens is now available fromApress The books covers the latest SQLite internals as wellas the native C interface and bindings for PHP Python PerlRuby Tcl and Java RecommendedCase Folding2006 jun 19 new book about sqlite the definitive guideto sqlite a new book by mike owens is now available fromapress the books covers the latest sqlite internals as wellas the native c interface and bindings for php python perlruby tcl and java recommended

2006 jun 19 new book about sqlite the definitive guideto sqlite a new book by mike owens is now available fromapress the books covers the latest sqlite internals as wellas the native c interface and bindings for php python perlruby tcl and java recommendedStemming2006 jun 19 new book about sqlite the definit guidto sqlite a new book by mike owen is now avail fromapress the book cover the latest sqlite intern as wellas the nativ c interfac and bind for php python perlrubi tcl and java recommend

2006 jun 19 new book about sqlite the definit guidto sqlite a new book by mike owen is now avail fromapress the book cover the latest sqlite intern as wellas the nativ c interfac and bind for php python perlrubi tcl and java recommendRemove Stop Words2006 jun 19 new book about sqlite --- definit guid-- sqlite - new book -- mike owen -- now avail --apress --- book cover --- latest sqlite intern -- well-- --- nativ - interfac --- bind --- php python perlrubi tcl --- java recommend

Posting Lists2006: doc52, doc871, doc1128, doc1137, doc2351jun: doc551, doc235119: doc88, doc92, doc93, doc1443, doc2351new: doc11, doc31, doc35, ., doc2337, doc2351book: doc192, doc331, doc409, ., doc2196, doc2351.recommend: doc1872, doc2351

QueriesRecommendation For A New BookCaseFoldingrecommendation for a new bookstemmerrecommend for a new bookStop wordsrecommend new book

Queriesrecommend new bookRetrieve posting lists foreach word in the queryrecommend: doc1872, doc2351new: doc11, doc31, doc35, ., doc2337, doc2351book: doc192, doc331, doc409, ., doc2196, doc2351T ake the intersection ofthe posting listsdoc2351

recommend new bookPosting List for“recommend”Posting List for“new”intersectintersectPosting List for“book”

OR Queries Simply take the union of the posting listsinstead of the intersection

recommend OR new OR bookPosting List for“recommend”Posting List for“new”unionunionPosting List for“book”

recommend book (sqlite OR tcl)Posting List for“recommend”Posting List for“book”Posting List for“sqlite”intersectunionintersectPosting List for“tcl”

NOT Queries Use the complement of the posting list Or better, use an “except” operator

book sqlite -pythonPosting List for“book”Posting List for“sqlite”intersectPosting List for“python”notHigh traffic linkintersect

book sqlite -pythonPosting List for“book”Posting List for“sqlite”Posting List for“python”intersectWorks just like theE XCE P T operatorin SQLexcept

Phrase Queries(naïve method) Do an ordinary AND query, then examine everydocument in the result set in a second pass andeliminate those that lack the phrase

Consider this phrase query: “the and but”

20061 jun2 193 new4 book5 about6 sqlite7 --- definit9 guid10-- sqlite12 - new14 book15 -- mike17 owen18 -- now20 avail21 -apress23 --- book25 cover26 --- latest28 sqlite29 intern30 -- well32-- --- nativ35 - interfac37 --- bind39 --- php41 python42 perl43rubi44 tcl45 --- java47 recommend482006: doc52:15, doc871:81, . , doc2351:1jun: doc52:16, doc2351:219: doc88:96, doc92:6, doc93:15, doc1443:31, doc2351:3new: doc11:7, ., doc2337:51, doc2351:4, doc2351:14book: doc192:33, ., doc2196:1, doc2351:5, doc2351:15, doc2351:25.recommend: doc1872:17, doc2351:48

The query“new book”Lookup posting listsfor each search termPhrase search combinerdoc11:7, ., doc2337:51,doc2351:4, doc2351:14A:BIf A Cand D B 1then A:Ddoc192:33, ., doc2196:1,doc2351:5, doc2351:15,doc2351:25C:Ddoc2351:5,doc2351:15

“new book” (sqlite OR tcl)Posting List for“new”Posting List for“book”Posting List for“sqlite”phraseunionintersectPosting List for“tcl”

Basic Operations1. Insert A New Document: Break the document up into words Append the document ID to the posting list for each word2. Perform A Query: Lookup the posting list for each word in the query Combine posting lists

What's so hard about that?

Appending DocIDs to Posting Lists Average email contains about 200 uniquewordsCompressed posting lists measure about 2030% of the size of the original documents.A multi-gibibyte index will not fit in cacheEach word lookup will require at least onerevolution of the disk platter: 8msTotal time to insert one email: 1.6 seconds

To get good performance, you need to keepyour working set smaller than our cacheThe working set for a full-text index can getreally big really fast

Keeping The Working Set Small Limit size of the lexicon–Stemming–Stop wordsCompress the posting lists aggressivelySpill stationary parts of posting lists into aseparate table that is not cached

Full Text Searchin

CREATE VIRTUAL TABLE email USING fts1(content);Looks and acts like a tablebut is not really a table.

CREATE VIRTUAL TABLE email USING fts1(content);Implemented using the “fts1”module.

CREATE VIRTUAL TABLE email USING fts1(content);Stores and indexes a singlecolumn named “content”.

CREATE VIRTUAL TABLE email USING fts1(“from” TEXT,“to” TEXT,subject TEXT,body TEXT);Able to index multiple columns.

CREATE VIRTUAL TABLE email USING fts1(“from” TEXT,“to” TEXT,subject TEXT,body TEXT,tokenize porter);Use the Porter stemmer.

db eval {INSERT INTO email(rowid,[from],[to],subject,body)VALUES( msgid, from, to, subj, body)}Insert just like a regular table

db eval {DELETE FROM email WHERE rowid msgid}Delete works the same, too

db eval {SELECT rowid, subject FROM emailWHERE email MATCH 'wyrick sqlite'}{# Display subject}Full-text search query using theMATCH clause.

db eval {SELECT rowid, subject FROM emailWHERE email MATCH 'wyrick sqlite'}{.}Table name as left operand meansmatch against any column of the table

db eval {SELECT rowid, subject FROM emailWHERE subject MATCH 'wyrick sqlite'}{.}Use a particular column name to limitthe search to that one column

db eval {SELECT rowid, subject FROM emailWHERE email MATCH 'from:wyrick sqlite'}{.}Qualifiers limit an individual search termto a particular column

Built-in snippet generatordb eval {SELECT rowid, snippet(email) FROM emailWHERE email MATCH 'from:wyrick sqlite'}{.}

FT S1 comes standardwith T CL bindings

Potential Uses Search for private websites Email Clients On-line documentation search Searchable history in web browsers Chatroom archive search Search version diffs in a CM system Text editors and IDEs

PervasiveFull-TextSearch

PervasiveFull-TextSearch QLite!edaMTgnisuyeasSdncl a

The Land Of Tcl/TkThe Territory of Pervasive Full-Text Search

The Land Of Tcl/TkThe Territory of Pervasive Full-Text Search

as the native c interface and bindings for php python perl ruby tcl and java recommended Case Folding. 2006 jun 19 new book about sqlite the definitive guide to sqlite a new book by mike owens is now available from ap