~~Title: Journal File Tables~~
~~NOTOC~~

<html><font color=#990000 size="+2"><b>Journal File Tables</b></font></html>

Journal Files Tables are log files that contain data for table change operations in transportable, platform-independent format. Journal File Tables are a critical component of StreamScape’s change capture mechanism. They can reside on the source or target server but exist outside of the database to ensure heterogeneity, simplicity and minimal impact to source data systems. 

Journal tables share data access and permissions.  They are opened for read and write simultaneously by the source system and by StreamScape’s data broker, allowing source systems to populate the contents in a transacted fashion using triggers.  A data fabric node continuously reads the change file and provides full SQL access to the transaction log content.  As such, users can configure Journal File Tables to be replication sources or write triggers that react to data changes that execute outside the source database. This architecture minimizes source system impact as there is no additional query or database I/O needed to enable the change capture process. 

Journal Tables move captured data to a replication queue managed by a broker node for delivery to target system(s).  In the event of an outage at the target the queue contains the most-recent data up to the point of the outage and will attempt re-delivery once target systems are online again.  Flexible recovery and restart options simplify exception handling automation.


        Starts and activates a Journal File Table collection.

Starting a JFT without options causes the engine to re-read the source file from beginning and 
fires triggers for each row being processed. Rows that are successfully processed have their 
Sequence Id and Timestamp stored as identity of the Last Processed Row. A Journal File ROW is 
considered processed if its Sequence Id and Timestamp are lower than the Id and Timestamp of 
the Last Processed Row. For example a Journal File ROW with Id 2012 and TS 5/22/18 13:55:01  
is considered processed if the Last Processed Row is Id 2015 and Timestamp 5/22/18 13:55:45  

WITH RECOVERY instructs the start operation to perform recovery from a specific point in the
Journal File. Without additional parameters this option functions the same as a START command.

FROM ID nnn tells the engine to start recovery from a specific Sequence Id. All prior content
is skipped.  Processing begins at the designated location in the file. Any triggers defined on 
the Journal File Table will fire only on processed rows being recovered. Providing a non-existent
Sequence Id results in no recovery and does not activate the Journal File Table.

FROM TIMESTAMP tells the engine to begin recovery from a particular point-in-time. The format
of the time stamp is dependent on how the JFT has been configured to interpret Timestamp values.
Use the DESCRIBE command to check this setting. ALTER the definition to change the format.

NO RECOVERY starts the Journal File Table and skips directly to the end of the file. The id for
the Last Processed Row is set to the last entry in the Journal File. Processing of data start
once the end of the file is reached.  Care should be taken when bypassing recovery.  On active 
systems unprocessed Journal Data may be skipped, resulting in data loss.  NO RECOVERY does not
support ID or TIMESTAMP modifiers.

SUSPEND ON FAIL instructs the engine to suspend Journal processing if there is a failure when 
processing a ROW.  Failures may occur due to Indexing issues, disk space problems or Malformed 
ROW contents.  Triggers may also throw explicit Exceptions that will suspend journal processing.
However noting can stop external writes to the Journal File.

SUSPEND ON FAIL will stop reading of source file and throw [advisory.journal.file.table.StateChange] 
if an error is encountered. 

To resume a Journal File use RESUME JFT command. The SUSPENDED state is not persistent.  After node 
restart a JFT is placed into active (resumed ) state automatically.  

Note that the JFT is NOT recovered on node startup.  It is RESUMED from Last Processed Row similar
to how NO RECOVERY functions.

A <code>[advisory.journal.file.table.StateChange]</code> event will be raised on each JFT state change.

<sxh DSQL; gutter: false;>

  START { JOURNAL FILE TABLE | JFT }  <Journal File Table Name> 
 	[ { WITH RECOVERY | NO RECOVERY } ] 
 		[ { FROM ID <SequenceId>  | FROM TIMESTAMP '<Timestamp>'  } ] 
 			[ SUSPEND ON FAIL ]
  
  Sample Synatax:
  
  start jft MyJournal with recovery from timestamp ‘5-22-18 13:55:45.000’ 
  
  The Timestamp format is controlled by the JFT’s file descriptor setting and shown as part of DESCRIBE command..

  More syntax examples:

  start jft MyJournal with no recovery 

  start jft MyJournal with recovery from id 201324  
  
</sxh>

So the basics are that you can simply START JFT so that you don’t have to type the whole thing out.. and you can restart recovery from the Beginning,
from a specific Sequence ID or Timestamp.  You can also start JFT without any recovery and just pick up from the last stored ID and Timestamp.  This is
similar to a resume command.. assuming that JFT contents have not completely been changed.  If we cannot find the position of the last processed row
based on ID and TS pair, that is an error and JFT is considered suspect.  We would need to reset or recover from a specific ID or TS..

If WITH RECOVERY is specified but ID or Timestamp are NOT specified, FULL Recovery is performed from start of file, just as if this option was NOT specified.
Full recovery occurs by default.  So the purpose of  WITH RECOVERY is only to allow users to specify where to start recovery from using  ID or TS.
Earlier ROWS are skipped.  Once recovery starts at the specific location, triggers are fired and the data are passed to the replication sub-system.

A Journal File ROW is considered processed if its ID and Timestamp are lower than the ID and Timestamp of the Last Processed Row.
For example a Journal File ROW with ID 2012 and TS 5/22/18 13:55:01  is processed if the Last Processed Row is ID 2015 and TS 5/22/18 13:55:45..  

If NO RECOVERY is specified we re-read the file to the end, Update the ID and TS to the last Journal File ROW and become Active.  All triggers are enabled 
after the Journal File activates.  FROM clause is not supported when NO RECOVERY is specified.


	Staring Journal With Full Recovery.

If the other options are provided:

		Staring Journal With Recovery from Sequence Id nnn.
		Staring Journal With Recovery from Timestamp MM-DD-YYYY HH:MM:SS.mmm
		Staring Journal With No Recovery. 

