.
\nIn the previous post we have seen how the OracleText index tables are maintained when new document arrives: At sync the new documents are read up to the available memory and words are inserted in the $I table with their mapping information. Now we will see how removed documents are processed. We will not cover updates as their are just delete + insert.
Here is the state from the previous post where I had those 3 documents:
\nwhich was synced alone, and then the two following ones were synced together:
\nThe $K is a IOT which maps the OracleText table ROWID to the DOCID (the fact that the primary key TEXTKEY is not at start is a bit misleading):
\nSQL> select * from DR$DEMO_CTX_INDEX$K;\n\n DOCID TEXTKEY\n---------- ------------------\n 1 AAAXUtAAKAAABWlAAA\n 2 AAAXUtAAKAAABWlAAB\n 3 AAAXUtAAKAAABWlAAC\nThe $R is a table mapping the opposite navigation (docid to rowid) storing a fixed-length array of ROWIDs indexed by the docid, and split into several lines:
\nSQL> select * from DR$DEMO_CTX_INDEX$R;\n\n ROW_NO DATA\n---------- ------------------------------------------------------------\n 0 00001752D000280000056940414100001752D00028000005694041420000\nThe $I table stores the tokens, the first 5 columns being indexed ($X) and the TOKEN_INFO blob stores detailed location of the token:
\nSQL> select * from DR$DEMO_CTX_INDEX$I;\n\nTOKEN_TEXT TOKEN_TYPE TOKEN_FIRST TOKEN_LAST TOKEN_COUNT TOKEN_INFO\n---------- ---------- ----------- ---------- ----------- ----------\nHELLO 0 1 1 1 008801\nWORLD 0 1 1 1 008802\nHELLO 0 2 3 2 0098010201\nMARS 0 3 3 1 008802\nMOON 0 2 2 1 008802\nWe have seen that the $I table can be fragmented for 3 reasons:
\nAnd the $N is empty for the moment:
\nSQL> select * from DR$DEMO_CTX_INDEX$N;\n\nno rows selected\nDo you remember how inserts are going to the CTXSYS.DR$PENDING table? Deletes are going to CTXSYS.DR$DELETE table:
\nSQL> delete from DEMO_CTX_FRAG where num=0002;\n\n1 row deleted.\n\nSQL> select * from CTXSYS.DR$DELETE;\n\nDEL_IDX_ID DEL_IXP_ID DEL_DOCID\n---------- ---------- ----------\n 1400 0 2\nI’ve deleted docid=2 but the tokens are still there:
\nSQL> select * from DR$DEMO_CTX_INDEX$I;\n\nTOKEN_TEXT TOKEN_TYPE TOKEN_FIRST TOKEN_LAST TOKEN_COUNT TOKEN_INFO\n---------- ---------- ----------- ---------- ----------- ----------\nHELLO 0 1 1 1 008801\nWORLD 0 1 1 1 008802\nHELLO 0 2 3 2 0098010201\nMARS 0 3 3 1 008802\nMOON 0 2 2 1 008802\nas well as their mapping to the ROWID:
\nSQL> -- $R is for rowid - docid mapping (IOT)\nSQL> select * from DR$DEMO_CTX_INDEX$R;\n\n ROW_NO DATA\n---------- ------------------------------------------------------------\n 0 00001752D000280000056940414100001752D00028000005694041420000\nHowever, the $N has been maintained to know that docid=2 has been removed:
\nSQL> select * from DR$DEMO_CTX_INDEX$N;\n\n NLT_DOCID N\n---------- -\n 2 U\nThis is the goal of $N (Negative) table which records the docid that should not be there and that must be deleted at next optimization (garbage collection).
\nFrom there, a search by words (‘normal lookup’) will give docid’s and rowid’s and the CTXSYS.DR$DELETE must be read in order to know that the document is not there anymore. It’s an IOT and the docid can be found with an index unique scan.
\nHowever for the opposite way, having a ROWID and checking if it contains some words (‘functional lookup’) we need to know that there is no document. In my case I deleted the row, but you may update the document, so the ROWID is still there. There is no pending table for that. It is maintained immediately in the $K table:
\nSQL> select * from DR$DEMO_CTX_INDEX$K;\n\n DOCID TEXTKEY\n---------- ------------------\n 1 AAAXUtAAKAAABWlAAA\n 3 AAAXUtAAKAAABWlAAC\nthe entry that addressed docid=2 has been deleted.
\nAll those changes were done within the same transaction. Other sessions still see the old values. No need to read CTXSYS.DR$DELETE for them. What I described above is only for my session: the normal lookup reading the queuing table, and the functional lookup stopping at $K. We don’t have to wait a sync to process CTXSYS.DR$DELETE. It’s done at commit:
\nSQL> commit;\n\nCommit complete.\n\nSQL> select * from CTXSYS.DR$DELETE;\n\nno rows selected\n\nSQL> select * from DR$DEMO_CTX_INDEX$R;\n\n ROW_NO DATA\n---------- ------------------------------------------------------------\n 0 00001752D000280000056940414100000000000000000000000000000000\nOf course we can’t read it but we see that part of it has been zeroed. That $R table is definitely special: it’s not stored in a relational way, and its maintenance is deferred at commit time.
\nBut nothing has changed in $I which contains garbage (and sync is not changing anything to that):
\nSQL> select * from DR$DEMO_CTX_INDEX$I;\n\nTOKEN_TEXT TOKEN_TYPE TOKEN_FIRST TOKEN_LAST TOKEN_COUNT TOKEN_INFO\n---------- ---------- ----------- ---------- ----------- ----------\nHELLO 0 1 1 1 008801\nWORLD 0 1 1 1 008802\nHELLO 0 2 3 2 0098010201\nMARS 0 3 3 1 008802\nMOON 0 2 2 1 008802\nAnd of course $N row is still there to record the deleted docid:
\nSQL> select * from DR$DEMO_CTX_INDEX$N;\n\n NLT_DOCID N\n---------- -\n 2 U\nI’ve not reproduced it here, but sync is not changing anything. Sync is for new documents – not for deleted ones.
\nWhat you need to remember here is:
\nOf course, you can sync at commit, but the second thing to remember is that
\nand both of them increase the size of the $I table and its $X index, making range scans less efficient. We will see more about that but the next post will be about queries. I’ve talked about normal and functional lookups and we will see how they are done. Let’s detail that.
\n", "protected": false } }