{ "date": "2015-10-11T20:52:46", "slug": "mapping-in-memory-column-store-to-datafile-row-store-extents", "link": "https://www.dbi-services.com/blog/mapping-in-memory-column-store-to-datafile-row-store-extents/", "title": { "rendered": "Mapping In-memory Column Store to datafile Row Store extents" }, "content": { "rendered": "

By Franck Pachot

\n

.
\nOracle In-Memory is an hybrid solution: an In-Memory Column Store in addition to the traditional Row Store.
\n\"CaptureIM01\"
\nIn the IMCS, data is stored in IMCU (In-memory compression units) and metadata is in SMU (Snapshot Metadata Units)
\nIn the row store, data is stored in datafile extents and metadata is stored in the dictionary (and in datafile header since Locally managed Tablespaces).
\nLet’s see how they map to eachothers.
\n

\n

Create table In Memory

\n

I’m creating a 2 million rows table in a LMT tablespace with autoallocate extent size.

\n
\ncreate table DEMO inmemory tablespace USERS as select rownum num,mod(rownum,10) ten from xmltable('1 to 2000000');\n
\n

and as in the previous post, I ensure that it’s populated in memory.

\n

V$IM_SMU_HEAD

\n

I can check the the IM Compression units from the SMU headers that show each IMCU with number of rows:

\n
\nSQL> select objd,tsn,startdba,extent_cnt,block_cnt,load_scn,itl_cnt,total_rows,invalid_rows from V$IM_SMU_HEAD;    \n                                                                                                     \n      OBJD        TSN   STARTDBA EXTENT_CNT  BLOCK_CNT LOAD_SCN            ITL_CNT TOTAL_ROWS INVALID_ROWS\n---------- ---------- ---------- ---------- ---------- ---------------- ---------- ---------- ------------\n    107958          6    1572875         22          5 941292582               255     491079            0\n    107958          6    1573890          8        126 941292582               255     561354            0\n    107958          6    1574914          8        126 941292582               255     524160            0\n    107958          6    1575938          7        126 941292582               255     423407            0\n
\n

You see that they reference extent information (DBA is Data Block Address – file number and block id).
\nIt’s not obvious here, but it’s relative DBA, with relative file number, and this is why you also have the tablespace number to absolutely identify a block in the database.

\n

DBA_EXTENTS and V$IM_TBS_EXT_MAP

\n

The Columns Store IMCUs are mapped to Row Store extents. One extent can populate several IMCUs but one IMCU contains data from only one extent. The mapping is available in V$IM_TBS_EXT_MAP
\nI show you how I join it to DBA_EXTENTS, using the START_DBA and END_DBA (and using dbms_utility to convert DBA_EXTENTS relative file number and block id to DBA). I get the data object id from DBA_OBJECTS:

\n
\nSQL> break on imcu_addr skip 1 duplicates\nSQL> select * from\n    (\n    select segment_name,file_id,block_id,blocks,bytes/1024/1024 MBytes\n     ,dbms_utility.make_data_block_address(relative_fno,block_id) extent_start_dba\n     ,dbms_utility.make_data_block_address(relative_fno,block_id+blocks-1) extent_end_dba\n     ,(select data_object_id from dba_objects where owner=user and object_name='DEMO' and object_type='TABLE') DATAOBJ\n    from dba_extents where owner=user and segment_name='DEMO'\n    ) e join (\n    select start_dba,end_dba,dataobj,to_char(imcu_addr,'0XXXXXXXXXXXXXXX') imcu_addr,len/1024/1024,to_char(smu_addr,'0XXXXXXXXXXXXXXX') smu_addr from V$IM_TBS_EXT_MAP\n    ) i on ( e.dataobj=i.dataobj and (i.start_dba between e.extent_start_dba and e.extent_end_dba) and (i.end_dba between e.extent_start_dba and e.extent_end_dba) )\n    order by 1,2,3;\n \nSEGMENT FILE_ID BLOCK_ID BLOCKS MBYTES _START_DBA _END_DBA DATAOBJ START_DBA END_DBA DATAOBJ IMCU_ADDR         LEN/1024/1024 SMU_ADDR\n------- ------- -------- ------ ------ ---------- -------- ------- --------- ------- ------- ----------------- ------------- -----------------\n \nDEMO          5  1572872      8  .0625    1572872  1572879  107958   1572875 1572879  107958  0000000067FFFDD8             7  0000000349F44770\nDEMO          5  1572880      8  .0625    1572880  1572887  107958   1572880 1572887  107958  0000000067FFFDD8             7  0000000349F44770\nDEMO          5  1572888      8  .0625    1572888  1572895  107958   1572889 1572895  107958  0000000067FFFDD8             7  0000000349F44770\nDEMO          5  1572896      8  .0625    1572896  1572903  107958   1572896 1572903  107958  0000000067FFFDD8             7  0000000349F44770\nDEMO          5  1572904      8  .0625    1572904  1572911  107958   1572905 1572911  107958  0000000067FFFDD8             7  0000000349F44770\nDEMO          5  1572912      8  .0625    1572912  1572919  107958   1572912 1572919  107958  0000000067FFFDD8             7  0000000349F44770\nDEMO          5  1572920      8  .0625    1572920  1572927  107958   1572921 1572927  107958  0000000067FFFDD8             7  0000000349F44770\nDEMO          5  1572928      8  .0625    1572928  1572935  107958   1572928 1572935  107958  0000000067FFFDD8             7  0000000349F44770\nDEMO          5  1572936      8  .0625    1572936  1572943  107958   1572937 1572943  107958  0000000067FFFDD8             7  0000000349F44770\nDEMO          5  1572944      8  .0625    1572944  1572951  107958   1572944 1572951  107958  0000000067FFFDD8             7  0000000349F44770\nDEMO          5  1572952      8  .0625    1572952  1572959  107958   1572953 1572959  107958  0000000067FFFDD8             7  0000000349F44770\nDEMO          5  1572960      8  .0625    1572960  1572967  107958   1572960 1572967  107958  0000000067FFFDD8             7  0000000349F44770\nDEMO          5  1572968      8  .0625    1572968  1572975  107958   1572969 1572975  107958  0000000067FFFDD8             7  0000000349F44770\nDEMO          5  1572976      8  .0625    1572976  1572983  107958   1572976 1572983  107958  0000000067FFFDD8             7  0000000349F44770\nDEMO          5  1572984      8  .0625    1572984  1572991  107958   1572985 1572991  107958  0000000067FFFDD8             7  0000000349F44770\nDEMO          5  1572992      8  .0625    1572992  1572999  107958   1572992 1572999  107958  0000000067FFFDD8             7  0000000349F44770\nDEMO          5  1573120    128      1    1573120  1573247  107958   1573122 1573247  107958  0000000067FFFDD8             7  0000000349F44770\nDEMO          5  1573248    128      1    1573248  1573375  107958   1573250 1573375  107958  0000000067FFFDD8             7  0000000349F44770\nDEMO          5  1573376    128      1    1573376  1573503  107958   1573378 1573503  107958  0000000067FFFDD8             7  0000000349F44770\nDEMO          5  1573504    128      1    1573504  1573631  107958   1573506 1573631  107958  0000000067FFFDD8             7  0000000349F44770\nDEMO          5  1573632    128      1    1573632  1573759  107958   1573634 1573759  107958  0000000067FFFDD8             7  0000000349F44770\nDEMO          5  1573760    128      1    1573760  1573887  107958   1573762 1573887  107958  0000000067FFFDD8             7  0000000349F44770\n \nDEMO          5  1573888    128      1    1573888  1574015  107958   1573890 1574015  107958  00000000686FFDD8             8  0000000349F54770\nDEMO          5  1574016    128      1    1574016  1574143  107958   1574018 1574143  107958  00000000686FFDD8             8  0000000349F54770\nDEMO          5  1574144    128      1    1574144  1574271  107958   1574146 1574271  107958  00000000686FFDD8             8  0000000349F54770\nDEMO          5  1574272    128      1    1574272  1574399  107958   1574274 1574399  107958  00000000686FFDD8             8  0000000349F54770\nDEMO          5  1574400    128      1    1574400  1574527  107958   1574402 1574527  107958  00000000686FFDD8             8  0000000349F54770\nDEMO          5  1574528    128      1    1574528  1574655  107958   1574530 1574655  107958  00000000686FFDD8             8  0000000349F54770\nDEMO          5  1574656    128      1    1574656  1574783  107958   1574658 1574783  107958  00000000686FFDD8             8  0000000349F54770\nDEMO          5  1574784    128      1    1574784  1574911  107958   1574786 1574911  107958  00000000686FFDD8             8  0000000349F54770\n \nDEMO          5  1574912    128      1    1574912  1575039  107958   1574914 1575039  107958  00000000626FFDA8             7  0000000339F44710\nDEMO          5  1575040    128      1    1575040  1575167  107958   1575042 1575167  107958  00000000626FFDA8             7  0000000339F44710\nDEMO          5  1575168    128      1    1575168  1575295  107958   1575170 1575295  107958  00000000626FFDA8             7  0000000339F44710\nDEMO          5  1575296    128      1    1575296  1575423  107958   1575298 1575423  107958  00000000626FFDA8             7  0000000339F44710\nDEMO          5  1575424    128      1    1575424  1575551  107958   1575426 1575551  107958  00000000626FFDA8             7  0000000339F44710\nDEMO          5  1575552    128      1    1575552  1575679  107958   1575554 1575679  107958  00000000626FFDA8             7  0000000339F44710\nDEMO          5  1575680    128      1    1575680  1575807  107958   1575682 1575807  107958  00000000626FFDA8             7  0000000339F44710\nDEMO          5  1575808    128      1    1575808  1575935  107958   1575810 1575935  107958  00000000626FFDA8             7  0000000339F44710\n \nDEMO          5  1575936    128      1    1575936  1576063  107958   1575938 1576063  107958  00000000620FFDA8             6  0000000339F34710\nDEMO          5  1576064    128      1    1576064  1576191  107958   1576066 1576191  107958  00000000620FFDA8             6  0000000339F34710\nDEMO          5  1576192    128      1    1576192  1576319  107958   1576194 1576319  107958  00000000620FFDA8             6  0000000339F34710\nDEMO          5  1576320    128      1    1576320  1576447  107958   1576322 1576447  107958  00000000620FFDA8             6  0000000339F34710\nDEMO          5  1576448    128      1    1576448  1576575  107958   1576450 1576575  107958  00000000620FFDA8             6  0000000339F34710\nDEMO          5  1576576    128      1    1576576  1576703  107958   1576578 1576703  107958  00000000620FFDA8             6  0000000339F34710\nDEMO          5  1576704    128      1    1576704  1576831  107958   1576706 1576764  107958  00000000620FFDA8             6  0000000339F34710\n
\n

I used sqlplus ‘break’ in order to separate the 4 IMCUs
\nIf you have partitioned tables, then you should add the tablespace number in the join.

\n

I have 45 extents following the auto extent size algorithm: 16x64KB extents totalizing 1MB, then 29x1MB extents.
\nThe ‘length’ is the size in the IMCS, compressed. Total is 28MB here.

\n

This mapping will help the re-population of a single IMCU: only the mapped extents have to be full scanned. And this is the way we can know which part of the table is populated or not.

\n

Currently, In-Memory Column store can be populated only from tables stored in the database, so the mapping with extents makes sense: full scan some extents in order to populate one IMCU. It’s possible that in future release we can populate IM from external tables. That would be a great feature for ETL datawarehouse loading. We will see then how it is mapped to the source. I hope to ear about that at OOW15.

\n", "protected": false } }