=== Reading and Writing Files ===

=== Working with File Large Objects (FLOBS) ===

Working with FLOBS should be like:

1. string s = readFile(…)  - should read into string instead of clob
2. byte[] b = readFile(…)  - should read into binary instead of blob
3. flob fl = linkFile(..)
    string/binary  s = fl; - should read flob and assign to variable
4. fromJSON(flob, … )  - should read flob automatically
5. fromJSON(readFile(…), … )  - file should be passed directly to serialized, without storing in local variable or on disk

linkFile(..)

readFile(..) 

writeFile(..)

 
=== Creating and Populating Tables with FLOB Data Type ===

FLOB data types can be specified as tuples (table columns) with storage at a spacific, secure location.  Keep in mind that a FLOB 
object is simply a pointer to an actual file in disk.  This is similar ot a BLOB object with the caveat that a FLOB is stored
external to the database.  In fact, because a FLOB is simply a pointer the actual file can be simultaneously refrenced and accessed
by the dataspace as well as any other application. FLOB resources are not locked.

This allows file resourcs and their data to be queried and manipulated via standard SQL without the need for complex programming.
FLOBs are stored in .dscahe(in default or user-specified location), independently of table type. Table only stores links to the inserted FLOB.

<sxh DSQL; gutter: true;>
create table Docustore(id int, key1 string, key2 string, document FLOB(location './data/docs'), primary key(id))
insert into Docustore values (2,'Zolpidem', 'Cancer', readFile('C:\Data Samples\Healthcare\NYC-FDA\drug-effect-docs\ambien\mao-main.pdf'))
select * from Docustore where key1 = 'Zolpidem'
flob f2 = select document from Docustore where id = 2;
select f2.size;
</sxh>

=== Accessing FLOB resources from a Browser ===

create function getFlobFile(int fileId) returns FLOB as
{
    return (select document from Docustore where id = @fileId limit 1);
}

And then to call this function and to open result as PDF we need the following URL
/ds/TSPACE.docs/fn/getFlobFile?fileId=2&responseFormat=pdf&responseMimeType=application/pdf&norowset=true


To get a table with links like 

^ id	^ key1		^ link ^
| 2	| Zolpidem 	| __Document__ |
| 3	| Prednisone	| __Document__ |


We need the following URL

/ds/TSPACE.docs/dsql?q=SELECT+Docustore%2Eid%2CDocustore%2Ekey1%2CtoUrl%28%28%28%27%2Fds%2FTSPACE%2Edocs%2Ffn%2FgetFlobFile%3FfileId%3D%27%2B+CAST%28Docustore%2Eid+AS+VARCHAR%2811%29%29%29%2B%27%26responseFormat%3Dpdf%26responseMimeType%3Dapplication%2Fpdf%26norowset%3Dtrue%27%29%2C%28%27link%27%29%29+as+link+FROM+Docustore+

This URL can be constructed with the following command

select toURL((select id, key1, toURL('/ds/TSPACE.docs/fn/getFlobFile?fileId=' + id + '&responseFormat=pdf&responseMimeType=application/pdf&norowset=true', 'link') as link from Docustore))

Currently toURL function work with SELECT queries only. That is why we need to specify URL for function call in toURL.

==== More fun with FLOBS ====

FLOBS can also be processed via Data Collections such as Tables or Queues:

<sxh DSQL; gutter: true;>
flob fl = linkFile('test.json’)
insert into flob_test values(3, fl)
string s = select f from flob_test where id = 3
map m = fromJSON(s, ‘map’)
</sxh> 

But also the following conversion should work
 
fromJSON(readFile(’test.json’), ‘map’)
fromJSON(linkFile(’test.json’), ‘map’)
fromJSON(fl, ‘map’)
string  s = fl;


Now ''readFile'' reads data into LOB/FLOB if the result of function is directly assigned to LOB/FLOB variable or column.
Additionally, if readFile is passed as first argument to fromJSON function, it also works without LOBs.

So now you can deserialize json from file using command like this

MyObject o = fromJSON(readFile(’test.json’), ‘MyObject’, …);

And it will be done in the optimal way.

Or you can read file into variable and then pass it to fromJSON

<sxh DSQL; gutter: true;>
string myjson = readFile(’test.json’, 0, -1, ‘utf-8');
MyObject o = fromJSON(myjson, ‘MyObject’, …);
</sxh>

You actually don’t need FLOBs in this case.
If result of readFile/linkFile function is assigned to FLOB variable(not directly inserted into table), then flob variable will contain only pointer to the file and actual read will be done once variable is inserted into table or access to file data is needed..

<sxh DSQL; gutter: true;>

// f contains a reference to file ‘test.json’
FLOB f = readFile(‘test.json’);  

// file ‘test.json’ is copied to dscache flobs location, real FLOB is created, 
// and reference to this flob is stored in the table
Insert into myflobs(1, f); 

// ‘test.json’ file is read and its data is passed to fromJSON, no FLOB is created
MyObject o = fromJSON(f, ‘MyObject’, …);   

</sxh>

linkFile is similar to readFile, except that linkFile function accepts the filename that is already in flob location, so it doesn’t copy file, just creates FLOB based on provided file.

Such postpone/lazy FLOBs creation was done to prevent orphan FLOBs, like it can occur with LOBs. 
For example if we read LOB into variable and don’t insert this variable into table, this LOB becomes orphan once variable goes out of scope.
