In Memory Hashtables in OpenInsight

Overview

Storing frequently accessed values in memory rather than on disk is a technique for improving your applications performance.  One useful tool is an in-memory hashtable, also known as key/value storage. The hashtable lets you write a value to a key and read it later.  OpenInsight offers several tools to help you hold values in memory.  Each tool has strengths and weaknesses.

 

Program

Description

Notes

RTP65

In memory hashtable, C dll

Fast.  No more than 100 tables.  Not designed for large number of Items.  No keys method.

RTI_HASHTABLE

A wrapper around RTP65

Same as RTP65, easier interface to program, adds keys  method

Revelation.RTIDotNetHashtable

A COM interface to the .Net Hashtable

Unlimited number of tables

Can store <idispatch> objects as well as strings.  Used by BFS connectors

RTI_HASHTABLE2

A Basic+ wrapper around the .Net hashtable

Same as Revelation.RTIDotNetHashtable, with easier interface

Scripting.Dictionary

A Microsoft hashtable

Ole object you can use for key value storage. Unlimited number of tables.  Not designed for large number of keys.

RTI_HASHTABLE3

A Basic+ wrapper around the Scripting Dictionary

 

Variable named commons

You can use variable named commons to implement a key/value sort of storage

Can store dimensioned arrays, multiple variables. Used by OpenInsight to implement window properties

 

See

RTP65

RTI_HASHTABLE

Revelation.RTIDotNetHashtable

Scripting.Dictionary

Variable named commons


RTP65Subroutine

Description

The RTP65 subroutine allows you to read and write key/value pairs to an in memory table.results.

Syntax

RTP65( Method, Handle, Key, Record, Reserved, Status )

 

Methods

 

0 - Create Table

1 - Open Table

2 - Close Table

3 - Clear Table

4 - Read Record

5 - Write Record

6 - Delete Record

 

You will want to make equates for these methods. For example

 

EQU CREATE_CACHE$  TO 0

EQU OPEN_CACHE$    TO 1

EQU CLOSE_CACHE$   TO 2

EQU CLEAR_CACHE$   TO 3

EQU READ_CACHE$    TO 4

EQU WRITE_CACHE$   TO 5

EQU DELETE_CACHE$  TO 6

EQU MAX_CACHE_CNT$ to 100

 

Method = Create table

rtp65 (CREATE_CACHE$, Handle,  tableName, '', '', Status)

 

Parameter

 

Description

Method

In

0

Handle

 

Null

Key

In

Name of the table to create

Record

 

Null

Reserved

 

Null

Status

Out

0 - Success

1 - Table already exists

5 - Max number of tables exceeded

 

 

Method = Open Table

rtp65 (OPEN_CACHE$, Handle,  tableName, '', '', Status)

 

Parameter

 

Description

Method

In

1

Handle

Out

Handle for the table

Key

In

Name of the table to open

Record

 

Null

Reserved

 

Null

Status

Out

0 - Success

2 - Table does not exist

 

 

Method = Clear table

rtp65 (CLEAR_CACHE$, Handle,  '', '', '', '')

 

Parameter

 

Description

Method

In

3

Handle

In

Handle from previously opened table

Key

 

Null

Record

 

Null

Reserved

 

Null

Status

 

Null

 

Clear erases all keys and values from the table. The table remains open.

Method = Read Record

rtp65 (READ_CACHE$, Handle,  Key, Record, '', Status)

 

Parameter

 

Description

Method

In

4

Handle

In

Handle from previously opened table

Key

In

Key of the record

Record

Out

Value of the record

Reserved

 

Null

Status

Out

0 - Success

3 - Invalid handle

4 - Table not open or Record not found

 

 

Method = Write Record

rtp65 (WRITE_CACHE$, Handle,  Key, Record, '', Status)

 

Parameter

 

Description

Method

In

5

Handle

In

Handle from previously opened table

Key

In

Key of the record

Record

In

Value of the record

Reserved

 

Null

Status

Out

0 - Success

3 - Invalid handle

4 - Table not open

6 - Memory Error

 

 

Method = Delete Record

rtp65 (DELETE_CACHE$, '',  tableName, '', '', Status)

 

Parameter

 

Description

Method

In

6

Handle

In

Handle from previously opened table

Key

In

Key of the record

Record

 

Null

Reserved

 

Null

Status

Out

0 - Success

3 - Invalid handle

4 - Table not open or Record not found

 

 

 

Remarks

RTP65 is simple and fast.  Before OI 9 is is limited to a total of 10 tables. As of 9.1 the limit is raised to 100 tabled. Openinsight uses a few tables itself.  RTP65 will slow down when there are a large number of keys.

 

 

 

Examples

 

 

 

Example 1

Subroutine RTP65_EXAMPLE(table)

/*

**  Demonstrate reading and writing cached recrods

*/

 

Declare Subroutine RTP65

$insert Logical

 

EQU CREATE_CACHE$  TO 0

EQU OPEN_CACHE$    TO 1

EQU CLOSE_CACHE$   TO 2

EQU CLEAR_CACHE$   TO 3

EQU READ_CACHE$    TO 4

EQU WRITE_CACHE$   TO 5

EQU DELETE_CACHE$  TO 6

EQU MAX_CACHE_CNT$ to 100

 

If assigned(table) Else table = ''

If table = '' then

    table = 'SYSPROCS'

End

               

Open table To f_table Else

                isOk = false$

                Return ''

end

 

* Create Cache

CacheName = "MYCACHE"

cache_status = 0

hCache = ''

 

rtp65 (CREATE_CACHE$, '',  CacheName, '', '', cache_status)

isOk = ( cache_status eq 0 )

If isOK then

                rtp65 (OPEN_CACHE$, hCache, CacheName, '', '', cache_status)

                isOk = ( cache_status eq 0 )

End

 

* put all the records in a cache

Call Rlist('SELECT ' : table, 5, '', '', '')

done = false$

ids = ''

Loop

                Readnext id Else done = true$

Until done Or Not(isOk)

                Read record From f_table, id Then

                                ids := id:@fm

                                rtp65(WRITE_CACHE$, hCache, id, record, '', cache_status)

                                isOk = ( cache_status eq 0 )

                end

repeat

 

* pull them all back out

col = ''

Loop

                Remove id From ids At col Setting mark

While id ne "" And isOK

                record = ''

                rtp65(READ_CACHE$, hCache, id, record, '', cache_status)           

                isOk = ( cache_status eq 0 )

repeat

 

rtp65 (CLEAR_CACHE$, hCache, '', '', '', cache_status)

 

Return

Example 2 - CACHE_MFS

Subroutine CACHE_MFS(CODE, BFS, HANDLE, NAME, FMC, RECORD, STATUS)

 

/* Name : CACHE_MFS

* Description:

* Cache records in lists

 

* Open -- put the cache number in the handle

* Read/Reado -- add the record to the cache

* write/delete/clear -- Update remote, local copy

* omnievent, FMC = 3 - clear cache

*

*

* Side Effects:

* Cache does not expire, so once read, will not refresh from disk

* Notes :

can call RTP57(OMNI.SCRIPT, BFS, HANDLE, NAME, CLEAR_CACHE$, RECORD, STATUS) clear the cache manually

Will not cache records with key[1,1] eq ‘%’

 

*/

 

$insert Logical

$insert FSErrors_100

$Insert FILE.SYSTEM.EQUATES

 

*$insert Cache_mfs_common

EQU CREATE_CACHE$  TO 0

EQU OPEN_CACHE$    TO 1

EQU CLOSE_CACHE$   TO 2

EQU CLEAR_CACHE$   TO 3

EQU READ_CACHE$    TO 4

EQU WRITE_CACHE$   TO 5

EQU DELETE_CACHE$  TO 6

EQU MAX_CACHE_CNT$ to 100

 

EQU CACHE_HANDLE_DELIM$ to \F7\

Equ CACHE_HANDLE_POS$ To 6

 

declare subroutine rtp65

 

common /cache_mfs_Common/init@,handle_cnt@,names@,file_handles@(max_cache_cnt$),cache_handles@,full_cache_flags@

cache_nr = Field(handle, cache_handle_delim$, cache_handle_pos$,1)

if cache_nr then

                transfer handle to hold

                handle = file_handles@(cache_nr)

                hCache = cache_handles@<cache_nr>

               

                $Insert File.System.OnGoSub

               

                transfer hold to handle

end else

                $Insert File.System.OnGoSub

end

 

Return

* -------------- Main Subs -----------------

READ.RECORD:

READO.RECORD:

RECORD = ''

cache_status = ''

if name[1,1] = '%' then

                GOSUB NEXT.MFS

end else

                rtp65(READ_CACHE$, hCache, NAME, RECORD, '', cache_status)

                If cache_status = 0 then

                                status = true$

                End else

                                * read from file

                                GOSUB NEXT.MFS                          

                                if status then

                                                * cache for next read

                                                rtp65(WRITE_CACHE$, hCache, NAME, RECORD, '', cache_status)

                                end

                End

end

Return

 

WRITE.RECORD:

GOSUB NEXT.MFS

if status then

                rtp65(WRITE_CACHE$, hCache, NAME, RECORD, '', cache_status)

End

Return

 

DELETE.RECORD:

GOSUB NEXT.MFS

rtp65(DELETE_CACHE$, hCache, NAME, '', '', cache_status)

Return

 

CLEARFILE:

GOSUB NEXT.MFS

rtp65(CLEAR_CACHE$, hCache, NAME, '', '', cache_status)

Return

 

DELETE.FILE:

GOSUB NEXT.MFS

rtp65(CLEAR_CACHE$, hCache, NAME, '', '', cache_status)

Return

* ---------------------------------------

* non-chained filing system calls

* ---------------------------------------

Flush:

Unlock.All:

Record = ""

Status = TRUE$

Return

 

Install:

if Assigned(init@) Else init@ = false$

If init@ else

                init@ = true$

                handle_cnt@ = 0

                mat file_handles@ = ''

                cache_handles@ = ''

                names@ = ''

                full_cache_flags@ = ''

end

Status = TRUE$

 

Return

* ---------------------------------------

* ---------------------------------------

* Chained Filing System Calls

* ---------------------------------------

LOCK.RECORD:

UNLOCK.RECORD:

GOSUB NEXT.MFS

RETURN

 

*--------------------------------------------------

SELECT:

READNEXT:

CLEARSELECT:

RECORD.COUNT:

* ---------------------------------------

CREATE.INDEX:

DELETE.INDEX:

UPDATE.INDEX:

SELECT.INDEX:

READNEXT.INDEX:

* ---------------------------------------

Omni.Script:

omniCall = If Assigned(FMC) then FMC<1> Else ''

If omniCall eq CLEAR_CACHE$ Then

   If hCache then

      rtp65 (CLEAR_CACHE$, hCache, '', '', '', cache_status)

   end

End else

   gosub Next.MFS

end

Return

Reserved:

* there is a critical error if this line is reached

Status = FALSE$

Return

* ---------------------------------------

OPEN.FILE:

GOSUB NEXT.MFS

if status then

                cache_status = ''

                locate name in names@ using @fm setting pos then

                                * File has already been opened once

                                * Is it the same file or a new one with same name.  The handles should tell.

                                * If the new handle (RECORD) is different, then the cached records are invalid

                                hCache = cache_handles@<pos>

                                orig_handle  = file_handles@(pos)

                                if record # orig_handle then

                                                rtp65 (CLEAR_CACHE$, hCache, '', '', '', cache_status)

                                end

                end else

                                hcache = ''

                                handle_cnt@+=1

                                if handle_cnt@ < = max_cache_cnt$ then

                                                * Create Cache

                                                rtp65 (CREATE_CACHE$, '',  NAME, '', '', cache_status)

                                                rtp65 (OPEN_CACHE$, hCache, NAME, '', '', cache_status)

                                                rtp65 (CLEAR_CACHE$, hCache, '', '', '', cache_status)

                                               

                                                * add it to the list

                                                handle_cnt@+=1

                                                names@<handle_cnt@> = name

                                                cache_handles@<handle_cnt@>=hCache

                                                file_handles@(handle_cnt@)= record

                                                full_cache_flags@<handle_cnt@>= false$

                                end

                end

                * embed the cache in RECORD, which is the handle of the file being opened

                if hCache then

                                record = FieldStore(record, CACHE_HANDLE_DELIM$, CACHE_HANDLE_POS$,1, handle_cnt@ )

                end

End

retval = status

Return

 

CREATE.FILE:

RENAME.FILE:

MOVE.FILE:

REMAKE.FILE:

* ----------------------------------------

Open.Media:

CREATE.MEDIA:

READ.MEDIA:

WRITE.MEDIA:

Close.Media:

gosub Next.MFS

Return

* ---------- End of Subroutine

* ==================================

* execute filing system chain

* ==================================

Next.MFS:

* Strips this MFS leaving the next fs as first element in array

FSList = delete(BFS, 1, 1, 1)

NextFS = FSList<1,1,1>

if len(NextFS) then

                call @NextFS(Code, FSList, Handle, Name, Fmc, Record, Status)

End

Return