Contents

Copyright Notice FairCom Typographical Conventions Introduction What is c-tree Plus? Sections of this Manual Quick Tour ISAM Quick Tour Introductory Tutorial Init Define Manage Done Additional Resources Low Level Introductory Tutorial Init Define Manage Done Additional Resources Using c-treeACE FairCom Operational Models Standard Client/Server Model Stand-alone Single-user Model Stand-alone Multi-user Model LOCLIB Model Multi-Threaded Stand Alone Model CTUSER Server Model Bound Server Model Custom Server Model Operational Model Issues Migrating Data Between Platforms and Operational Models Single-user Standalone Multi-user standalone Client/server File Migration Migrating Your Application Between Operational Models Single-user to Multi-user Standalone or Client/Server Multi-user Standalone to Client/Server Adding Transaction Processing Single-threaded to Multi-Threaded c-treeACE Concepts “C” Concepts Data Type Conventions Strings Prototyping c-tree Plus Keys Alternative Key Types Fixed Length Keys Leading Character Compression Keys Padding Compression Keys Combined Compression Keys ISAM Keys Duplicate Keys Sequence Numbers Data Records Buffer Regions IMPORTANT - Data Record Delete Flag Resource Records in Fixed Length Files Data and Index Files File Formats Extended Feature Support Operating System File Size Limits Huge File Support Huge File Basics Xtd8 File Creation Functions Huge File Creation Example Record Offsets Under Huge File Support Segmented File Support Segmented File Basics Automatic Segments SEGMDEF Structure SetFileSegments Function File Segment Example Partitioned File Support File Modes Virtual File Open Fixed versus Variable Length Records Multi-User File Mode Transaction Mode Mirroring Skip Mode I/O Management Resources Superfiles Corrupt Files filmod Values File Mode Relationships Default File Modes Permanent File Modes Temporary File Modes Read-only and Shared File Modes Extended File Creation Block Structure Extended File Modes (x8mode) Host Segment Size (segsiz) Maximum Number of Segments (segmax) Max File Size Limit (mxfilzhw, mxfilzlw) File Extent Size (lxtsiz, fxtsiz) Individual File Disk Space Threshold (dskful) File Encryption Key (filkey) Relative Partition Key Number (prtkey) Special Cache Value Parameter (splval) Data and Index File Numbering Data File Extension Low-Level Functions ISAM Functions Transaction Processing Overview ctNOGLOBALS Concepts Default Temporary File Path in Standalone and LOCLIB Models Error Handling File Recovery c-tree Plus Constraints File Size Record Size Serial Number Segments Transaction High-Water Marks Enforce Maximum Disk Read/Write Sizes on Windows ISAM Functions A Simple Application Application Outline ISAM Parameters The Application Initialize c-tree Plus and open files Add a record Find a record Delete a record Close the system ISAM Error Handling Current ISAM Record ISAM Function Overview Incremental ISAM System Functions ISAM Parameter File System Functions Data File and Index Functions Set Functions ISAM File Relationships Incremental ISAM Structures IFIL Structure IIDX Structure ISEG Structure Key Segment Modes IMPORTANT - Key Value Assembly TransformKey CurrentISAMKey Sequence Number Segments Descending Key Segment Values Alternative Collating Sequence END of Key Segment RECBYT Segment Mode Improved Deleted Space Management Incremental ISAM Example Incremental ISAM and Resources ISAM Parameters ISAM Parameter File Organization Parameter File Contents Initialization Record Data File Description Record Index File Description Record Optional Index Member Record Key Segment Description Record ISAM Examples Fixed Length Parameter File Examples Variable-Length Parameter File Example Multiple Data File Parameter Setup Record Schemas and DODA Resources Record Schema Schema Map Schema Name Details Record Structure Field Types Create the Record Schema with PutDODA Key Segments Multi-User Concepts Multi-user Interference Multi-user Updates Multi-user Non-Server Mode Rebuild File Locking Multi-user File Mode Record Locking Checklock ISAM Level Data Record Locks Using LockCtData with LockISAM LockISAM and Resources Recursive Locking Support Diagnosing Lock Issues Low Level Data Record Locks c-treeACE Features Resources Resources and Files Resource Identification Disabling Resources Resource Functions Resource Example FairCom Defined Resources Alternate Collating Sequence File Security Padding and Delimiter Values DODA and Record Schema IFIL information Resource Locks Superfiles Creating/Opening Superfiles and Members/Host Superfile Members/Host Closing Superfiles and Members Restrictions/Considerations Nested Superfiles Physical Order Node Size Detect if Superfile Member Names are too Long Temporary Files Rebuild Single System File Descriptor Risks Batched Operations Batch Parameters Partial Key Request structure First Call Verify Option Result Option Lock Option Completion option Continuation Calls Complete or Cancel Call Batch Record Insertion Batch Read in Physical Order Batch Read in Key Decreasing Order Batch Read with Index Range Definition Additional DoBatchXtd() Options Performance Enhancement Example Use of DoBatchXtd() Index Ranges Index Ranges Overview Using Index Ranges from the c-tree ISAM API Allocating an Index Range Reading Records Using an Index Range Freeing an Index Range Blocking Record Reads Using Blocking Record Reads Data Filters and Conditional Indexes Data Filters Conditional Index Support Conditional Expression Parser Conditional Expression Syntax Constants Variables Parentheses Predefined Functions Type Casting Automatic Type Promotion Operators Variable-length Records with Conditional Expressions Conditional Expression Callback Function Other Uses for the Expression Parser Parsing your expression Evaluating your expression Multi-threaded API Introduction to Threads Some Thread Fundamentals FairCom Thread Experience Thread API Summary ctThrd Function Overview Thread Management Functions Thread Communication Thread synchronization Thread API Details Examples ctmtap.c and wtmtap.c - Multi-threaded testing programs ctmtex.c - Multi-threaded testing program Multiple ISAM Contexts (Positions) Rebuild/Compress Utility Callback Feature Portable Data Through UNIFRMAT Support Modes Automatic Mode Manual Mode UNIFRMAT Server Limitations Data Alignment Multiple Instance Support ctNOGLOBAL Basics Automatic Windows 3.1 Instance Support Pre-Function Capability Performance Optimization sect optimization Index cache size Data cache size Rebuild and Compact Optimization Single-user Standalone Index Cache Sort Files NO_IDXENT Faster ISAM Access Disable key buffer support Enable key buffer support Considerations Enhanced Precision for International Languages Client/Server Support Standalone Support Helpful Internal Functions File System First/Next Internal Disk Space Available function Internal wildcard matching logic Date routine added to ctsadm Sample Programs Simple Applications Simple File Maintenance Initialize the system Error Handling Adding Data Delete a key Close the system Same Application with ISAM functions ISAM parameter file Error handling Adding Data Delete record Close the system Using Incremental ISAM structures Incremental ISAM structure Initialize the system File Searches and Sets Listing a file Sets Scanning, Updating, Deleting Start the scan Scanning the file Update the record. Delete a record Windows specific samples wtixmg - A Windows version of the ctixmg sample wtlxmg - A Windows version of the ctlxmg sample wtmtap - Windows Multi-threaded Sample Application Sample Programs ctcnfg ctexmg - ISAM parameter file database sample ctixmg - Incremental ISAM / Resource sample ctlxmg - LOCLIB Sample Program ctmark - Simulated data generation program ctmtap - Multi-threaded API sample and performance tester ctmtex ctsxmg - Incremental ISAM / Record Schema example ctvxmg - Variable record length ISAM parameter file example lowlevel, isam, isam1, isam2, isam3 ioperf selomt trnhis CGI JAVA Visual Basic c-treeACE Utility Programs ctcmpc - File compaction ctclntrn Utility - Clean Transaction Mark ctcmpcif - IFIL-based Compact Utility ctcv43 - Convert V4.3 Data File to c-tree Plus Format ctcv67 - Extended File Conversion Utility ctdidx - Create flat key file ctdmpidx - Index Dump Utility ctexmc - File creation utility / example ctfileid - Update File IDs ctflat - Create c-tree Plus file from a flat file ctflvrfy - Index Verify Utility cthghtrn - Displays the high-water mark for transactions ctidmp - Examine Dump Files ctin43 - Create flat key file from V4.3 index file ctindx - Create index file from flat key file ctinfo - Incremental ISAM utility ctmtlk - Lock operation test utility ctmove - Moving Files without Stopping the c-tree Server ctpkey - Print key utility ctpotp - Parameter to OTP conversion ctptoi - Parameter to IFIL conversion ctrbld - Parameter file rebuild utility ctrbldif - IFIL-based Rebuild Utility ctsbld - Superfile rebuild pre-pass ctsbldm - Rebuild Superfile Index Members ctscmp - Superfile Compact Utility ctsfex - Superfile Data Export Utility ctsfil - Superfile contents ctstat - Statistics Utility (emb) Admin-System Report Example Tivoli-System Report Example Admin-File Report Example Tivoli-File Report Example Admin-User Report Example Function Timing Report Example Text Report Example I/O Time Statisics Example I/O Statisics per File Example Existing Connections Userinfo Example ISAM Statistics Example Enable Function Call Times by File Function call Times by File Example Memory File Usage Example cttrap - Communications Trap Playback utility cttrnmod - Change Transaction Mode Utility ctunf1 - File Reformatting Utility ctunf2 - UNIFRMAT file reformatting utility ctupdpad - Update the c-treeDB Pad Resource Utility ctvlqa - Variable-length quality assurance utility ctvfyidx - Index Verify Utility Standalone Multi-User Node Locks Data Record Lock Implementations Extended Lock Strategy Complications with Superfiles in Standalone Multi-user Models Complications with ctLOCK_TOP Dummy Lock File Exclusive File Opens EXCLUSIVE File Caching c-treeACE Database Engine Server Advantages FairCom Client/Server Communication Server-Application Relationship Logging On and Off the Server Calculating Memory Usage Controlling Server Memory Calculating File Storage Space Copying Server Controlled Files Multiple Server Support per Client Local Library Support Server Utilities ctadmn - Server Administrator Utility ctstop - Server Stop Utility sa_admin - Command-line Server Administrator Utility ctdump - Schedule a Dynamic Dump ctrdmp - Dynamic Dump Recovery or System Rollback ctfdmp - Forward Dump Utility ctldmp - Transaction Log Dump Performance I/O caching SetOperationState Batch Operations ISAM Functions ctDUPCHANEL File Mode Fastest Server Communication Protocol Optimizing Transaction Processing - ADVANCED Transaction Control Background Transaction Options Transaction Log Flush Delay Resource Records Detected File Modes Record Locking mode I/O Management Server Insight Server Threads Server Signals Client/Server ctntio Communications Errors (Formerly VDP Errors) Server Administration API Deployment Options Tamper-Proof Server Configuration File Encryption of the FAIRCOM.FCS Validation of FAIRCOM.FCS Disk Full Detection File-by-file Disk Full Checks Volume Disk Full Checks Server System Event Log Retrieving System Event Log Entries Adding System Event Log Entries System Event Log evclass and event Codes Multi-CPU Performance COMPATIBILITY NLM_DEFER_THREADSWITCH Diagnostic Tools cttrap - TRAP_COMM Utility cttrap - Communications Trap Playback utility DIAGNOSTICS LOWL_FILE_IO Keyword Diagnostic Dump of Server File-lock Table DIAGNOSTICS USER_FUNCTIONS keyword added Dynamic CRC Check support for SPX and TCP/IP c-tree Server Cache Sub-System Allocating Memory for Data/Index Caches Cached Data - Writing to Disk Advanced Cache Settings Buffer Run Length Multi-page cache Disabling File Cache Dedicated File Cache Limiting Special Cache Space Priming Cache Avoiding file flush at server shutdown Other Notes SystemConfiguration Cache/Buffer Statistics Create a List of Arbitrary Names in Server Configuration File Application Name List Usage Application Name List Restrictions Application Name List Example Server Performance Monitoring Performance Monitoring Overview Performance Monitoring Using the ctstat Utility Performance Monitoring Using Server Keywords Automatically Logging Performance Snapshots Automatic Logging to the Server System Event Log Automatic Logging to SNAPSHOT.FCS Performance Monitoring Using the SnapShot API SnapShot API Function Usage Automatic Snapshot Support Function Timing Support System Snapshot Support User Snapshot Support File Snapshot Support Combined Snapshot Support Snapshot Contents System Snapshot Structure User Snapshot Structure File Snapshot Structure Function Timing Memory Files Memory Files Overview Using Memory Files Creating Memory Files Using Server Configuration Keyword Creating Memory Files Programmatically Sharing Memory Files Created Programmatically Collecting Memory File Statistics Memory File Limitations Tip: Faster Server Shutdown with Memory Files Server-Based Queues Server-Based Queues Overview Using Server-Based Queues Creating or Opening a System Queue Writing to a System Queue Reading from a System Queue Retrieving the Size of the Next Message in a System Queue Retrieving the Number of Messages in a System Queue Closing a System Queue File Notification File Notification Overview Using File Notification Enabling Notification for Actions on a File Open Files for Which Notification is Requested Open a System Queue Establish Notification for the File Receiving Notifications for Actions on a File Notification Queue Message Format Fixed Portion of Notification Queue Message Optional Key Values Optional Actor Node Name Optional Full Record Image Reading Notification Messages Disabling Notification for Actions on a File Notification Callbacks Partitioned Files Highlights Overview Implementation Partition Naming Rules Advanced - Maximum Partitions Operation Raw Partition Numbers Unique Keys Serial Segments (SRLSEG) Transaction Processing Encryption Partitioned File Security - File password support Partition Administration Function Client-Side Operation Introduction c-tree Plus Client Side Installation/Operation Compatibility c-tree Plus Parameter Files Data Compatibility Client Side Specifics Max Files (Parameter Files Only) Modifying ctoptn.h Moving Data Files Server/Host Naming Convention Unix Communications Notes Client TCP/IP Connect and Communication Timeout Options Server Name Broadcast Feature File Security and Encryption Accessing File Security Forms of System Security Users, Groups, and Passwords File Permissions File Passwords User Profile Automatic TransformKey Save Current ISAM record Allow ADMIN Access When Max User Limit Hit Logon Control Options Logon Strike-Out Options Suspend Server - Block User Logins Suspend the Server Server Keyword: STARTUP_BLOCK_LOGONS Encryption Standard File Encryption File-specific Encryption Control Custom Encryption Options TCP/IP Encryption Data Integrity Transaction Processing Concepts Atomicity Automatic Recovery Why do we need Transaction Processing? Basic Transaction Processing Create Files Transaction mode Begin Transactions End Transaction Two-Phase Transactions File Modes Record Locking Begin Commit Marking Transactions Begin Commit Aborting a Transaction Savepoints Errors in Transactions File Operations During Transactions Closing Files During Transaction Processing Optional Defer of Close Until Transaction Commit/Abort Defer File Delete Transaction Dependent Creates and Deletes Automatic Commits Transaction Processing Logs Automatic Log Adjustments Automatic Log Size Adjustment Automatic Increase of Active Transaction Logs SystemConfiguration Log Space Reporting SystemConfiguration Log Reporting Enhancements Automatic Recovery Transaction High-Water Marks Single-User Transaction Processing Transaction Processing On/Off Single User Transaction Processing Control Clear Transaction Logs Log Paths Additional Single-User Transaction capabilities Single-user Transaction processing hard coded file zero conflict Transaction History TransactionHistory Basic Operation TransactionHistory Output TransactionHistory Advanced Operation Optional Record Header Multiple History Sets Mirroring Recovery Capabilities Miscellaneous Points MIRROR_DIRECTORY Server Keyword Error Returns Limitation Unicode Support Unicode Concepts Preparation Storing Unicode Data Storing UTF16 Data Storing UTF8 Data Unicode Key Segments Unicode Libraries Required for c-tree Plus ICU - International Components for Unicode How to Specify an ICU Unicode Key Segment Extended Key Segment Definition ICU Collation Option Overview Extended Key Segment Definition Example Extended Key Segment Default Hierarchy Extended Key Segment API API Example Error Codes Server Configuration Keywords For Unicode Segment Default Unicode File Names Mirrored File Names c-treeACE Server SDK Overview Making your own Server STEP 1: Install the c-tree Plus source code to your machine. STEP 2: Build the makefile with the FairCom mtmake utility. STEP 3: Execute the makefile to build your Server. Make Summary Server-Side Models Create a c-tree Server - FairCom Custom Server User-Defined Callback Functions for File Create, Open and Close Operations Create your own Engine - Bound Server Model Quick Start Install Make Test Working with Your Mainline Initialize Threading at Startup Set Configuration at Threading Initialization Initialize Mutexes and Blocks Use FairCom Threading API Each Thread is a Separate Instance of c-tree Plus Release threading at shutdown Bound Server and Custom Server Advanced Server SDK Options CTUSER Function Default Implementation Simple Example Advanced File Encryption Advanced File Encryption Development Server Defaults c-tree Client Implementation Advanced Encryption Configuration ctcpvf Utility - Create Master Password Verification File Client Operation Cryptographic techniques Secure One-Way Hash Functions MD5 Hash Function Block Ciphers Data Encryption Standard - DES Triple DES with Two keys - 2DES Triple DES with Three keys - 3DES Advanced Encryption Standard - AES (Rijndael) Electronic CodeBook Mode - ECB Cipher Block Chaining Mode - CBC Output FeedBack Mode - OFB Pseudo-Random Number Generator Random Bit Stream Generator Random Seed printf() and exit() Overrides User-defined SIGNAL_READY and SIGNAL_DOWN Functions Conclusion c-treeACE Error Codes c-tree Plus Error Code Reference 2xx Error Codes 471-496 Sort Error Codes 9xx Error Codes Function Name Cross Reference Full Names Abbreviated (short) Names Function API Listing Initialization API ctThrd API Instance Control API ISAM Initialization API Low-level Initialization API Data Definition API ISAM Data Definition API Low-level Data Definition API Data Manipulation API ISAM Data Manipulation API Batch API Context API Sets API Low-level Data Manipulation API Utility Functions Server Administration API Transaction Processing API Upgrade Previous Editions Upgrading V6 Applications Duplicate Keys #defines Converting c-tree V4.1F-V4.3C Applications Application Conversion Details Backward Compatibility Using Low-Level Functions Simple Application Application Outline The Application Initialize c-tree Plus Add a record Find a record Delete a record Close the system Low-Level Function Overview System Functions File Manipulation Functions Data Record Manipulation Key Manipulation Transaction Processing Functions Resource Functions Common Entry Point Functions Forced Single Entry Point Capability Extended Feature Parameter Blocks Logon Block ISAM Block IFIL Block Create File Block Open File Block Key Estimate Block Function Calls c-tree Plus Function Descriptions Abort AbortXtd AddCtResource AddKey AddRecord AddVRecord AllocateBatch AllocateRange AllocateSet AvailableFileNbr Begin BuildKey BlockingISAMRead ChangeBatch ChangeHistory ChangeISAMContext ChangeSet CleanIndexXtd ClearSavePoint CloseCtFile CloseCtFileByName CloseIFile CloseISAMContext CloseRFile Commit CompactIFile CompactIFileXtd CompactIFileXtd cpybuf CreateDataFile CreateDataFileXtd CreateDataFileXtd8 CreateIFile CreateIFileXtd CreateIFileXtd8 CreateIndexFile CreateIndexFileXtd CreateIndexFileXtd8 createindexmember.htm CreateISAMXtd ctCallback ctFILELIST ctFILBLK ctFileCloseCallback ctFileCreateCallback ctFileOpenCallback ctFILMTX ctfiltercb_rowl ctGETHGH ctLOKTIMOUT ctMBprefix ctNotify CtreeAsynchronous CtreeCheckPoint CtreeFlushFile CtreeFlushFileXtd CtreeUserOperation CtreeUserOperation CreateISAM ctSETHGH ctSetOWNER ctSQLImportTable ctSysQueueClose ctSysQueueCount ctSysQueueLIFOWrite ctSysQueueMlen ctSysQueueOpen ctSysQueueRead ctSysQueueWrite ctThrdAttach ctThrdBlockCls ctThrdBlockGet ctThrdBlockInit ctThrdBlockRel ctThrdBlockWait ctThrdCreate ctThrdData ctThrdDataSet ctThrdDetach ctThrdExit ctThrdHandle ctThrdInit ctThrdLIFOWrite ctThrdMutexCls ctThrdMutexGet ctThrdMutexRel ctThrdMutexTry ctThrdQueueClose ctThrdQueueCount ctThrdQueueMlen ctThrdQueueOpen ctThrdQueueRead ctThrdQueueReadDirect ctThrdQueueWrite ctThrdQueueWriteDirect ctThrdSemapCls ctThrdSemapGet ctThrdSemapInit ctThrdSemapRel ctThrdSemapTry ctThrdSleep ctThrdTerm ctTransferFile ctu16TOu8 ctu8TOu16 ctUPDICU CurrentFileOffset CurrentISAMKey CurrentLowLevelKey DeleteCtFile DeleteCtResource DeleteIFile DeleteKey DeleteKeyBlind DeleteRecord DeleteRFile DeleteVRecord DoBatch DoBatchXtd DropIndex EnableCtResource EstimateKeySpan FindFileByName FirstInRange FirstInvRange FirstInSet FirstInVSet FirstKey FirstRecord FirstVRecord FreeBatch FreeBatchNbr FreeHistory FreeHistoryNbr FreeRange FreeSet FreeSetNbr GetAltSequence GetConditionalIndex GetCtFileInfo GetCtreePointer GetCtResource GetCtTempFileName GetDODA GetGTEKey GetGTERecord GetGTEVRecord GetGTKey GetGTRecord GetGTVRecord GetIFile GetKey GetLTEKey GetLTERecord GetLTEVRecord GetLTKey GetLTRecord GetLTVRecord GetORDKey GetRecord GetSerialNbr GetServerInfo GetServerInfoXtd GetSuperFileNames GetSymbolicNames GetVRecord GetXtdCreateBlock GetXtdKeySegmentDef InitCTree InitCTreeXtd InitISAM InitISAMXtd IOPERFORMANCE IOPERFORMANCEX KeyAtPercentile LastInRange LastInVRange LastInSet LastInVSet LastKey LastRecord LastVRecord LoadKey LockCtData LockDump LockISAM LockList NbrOfKeyEntries NbrOfKeysInRange NbrOfRecords NewData NewVData NextCtree NextInRange NextInVRange NextInSet NextInVSet NextKey NextRecord NextVRecord OpenCtFile OpenCtFileXtd OpenFileWithResource OpenFileWithResourceXtd OpenIFile OpenIFileXtd OpenISAM OpenISAMContext OpenISAMXtd PartitionAdmin Perform PermIIndex PermIIndex8 PositionSet PositionVSet PreviousInRange PreviousInVRange PreviousInSet PreviousInVSet PreviousKey PreviousRecord PreviousVRecord PutDODA PutIFile PutIFileXtd PutIFileXtd8 PutXtdKeySegmentDef QuietCtree ReadData ReadIsamData ReadIsamVData ReadVData RebuildIFile RebuildIFileXtd RebuildIFileXtd8 RebuildIIndex RegisterCtree ReleaseData ReleaseVData RenameFile RenameIFile RenameIFileXtd ReplaceSavePoint ReReadRecord ReReadVRecord ResetRecord RestoreSavePoint ReWriteRecord ReWritePartialRecord ReWriteVRecord SA_ERRMSG SA_FILES SA_GROUP SA_LOGOF SA_LOGON SA_USERS SA_WILDCARD Security SetAlternateSequence SetCallbackOnRebuild SetDataFilter SetEncryption SetFileSegments SETLOGPATH SetNodeName SetOperationState SetRecord SetSavePoint SetSystemConfigurationOption SetVariableBytes SnapShot StopServer StopServerXtd StopUser StopUserAsync SuperfilePrepassXtd SwitchCtree SystemConfiguration SystemLog SystemMonitor TempIIndexXtd TempIIndexXtd8 TestFileNbr TestHugeFile TRANRDY TransactionHistory TransformKey TransformSegment TransformXtdSegment UnRegisterCtree UpdateConditionalIndex UpdateCtResource UpdateFileMode UpdateHeader UserLogEntry uTFRMKEY VDataLength VRecordLength vtclose WhichCtree WriteData WriteVData Index