RAD Studio 10.4.1 is now available! Learn more. Looking for discounts? Visit our Special Offers page!
News

API Limits with #FDEC

API Limits with FireDAC Enterprise ConnectorsThe FireDAC Enterprise Connectors (#FDEC) by CData and Embarcadero make it really easy to work with various APIs just like you would any SQL database. For example if you want to publish the results of a query to a Google Sheet (which I find incredibly useful) then it is just a few FireDAC components and you are off to the races. You might run into an API limit though.

What is an API limit? Most rest services have a limit to how often a client can call a specific API within a certain amount of time. Google calls this their usage limit:

This version of the Google Sheets API has a limit of 500 requests per 100 seconds per project, and 100 requests per 100 seconds per user. Limits for reads and writes are tracked separately. There is no daily usage limit.

That may seem like a lot, but I found I was running into that limit pretty quick once I moved my project into production. Luckily FireDAC and the FireDAC Enterprise Connectors have a simple workaround: Batch Processing.

Using the Array DML features of FireDAC you can batch multiple DML (Data Manipulation Language) operations into a single API call. The FDEC Google Sheets documentation from CData doesn’t cover Array DML, but the component supports this (they are updating the documentation). The Elasticsearch documentation does cover Batch Processing with an example, and I’ve used this with Sheets and it works great!

Bulk Insert

The following example prepares a single batch that inserts records in bulk.

FDConnection1.ResourceOptions.ServerOutput := True; FDQuery1.SQL.Text := ‘insert into Account values (:Name, :Id )’; FDQuery1.Params.ArraySize := 100; FDQuery1.Params[0].AsStrings[0]:= ‘MyName1’; FDQuery1.Params[1].AsStrings[0]:= ‘MyId1’; //next statement FDQuery1.Params[0].AsStrings[1]:= ‘MyName2’; FDQuery1.Params[1].AsStrings[1]:= ‘MyId2’; … FDQuery1.Execute(FDQuery1.Params.ArraySize); ShowMessage(IntToStr(FDQuery1.RowsAffected));

To retrieve the Ids of the new records, query the LastResultInfo#TEMP table:

sName := FDQuery1.Open(‘SELECT * FROM [LastResultInfo#TEMP]’);

Bulk Update

The following example prepares a single batch that inserts records in bulk.

FDQuery1.SQL.Text := ‘update Account set Name = :Name WHERE Id = :Id’; FDQuery1.Params.ArraySize := 100; FDQuery1.Params[0].AsStrings[0]:= ‘Floppy Disks’; FDQuery1.Params[1].AsStrings[0]:= ‘Id1’; //next statement FDQuery1.Params[0].AsStrings[1]:= ‘Jon Doe’; FDQuery1.Params[1].AsStrings[1]:= ‘Id2’; … FDQuery1.Execute(FDQuery.Params.ArraySize); ShowMessage(IntToStr(FDQuery1.RowsAffected));

Bulk Delete

The following example prepares a single batch that inserts records in bulk:

FDQuery1.SQL.Text := ‘delete Account where Id = :Id’; FDQuery1.Params.ArraySize := 100; FDQuery1.Params[0].AsStrings[0]:= ‘MyId1’; //next statement FDQuery1.Params[0].AsStrings[1]:= ‘MyId2’; … FDQuery1.Execute(FDQuery.Params.ArraySize); ShowMessage(IntToStr(FDQuery1.RowsAffected));

If you want to learn more about Array DML check out these videos:

Array DML Skill Sprint with Pawel Glowacki

 

FireDAC in Depth with Cary Jensen

 

Also check out Cary Jensen’s book on the topic of FireDAC in Depth.


Reduce development time and get to market faster with RAD Studio, Delphi, or C++Builder.
Design. Code. Compile. Deploy.
Start Free Trial   Learn More About Upgrading

About author

Chief Developer Advocate for Embarcadero Technologies.
Related posts
C++NewsRAD Studio

Open for Business with 10.4! - May 2020 GM Update

Delphi

SQLGate Validates Delphi As The Right Tool For Building High Performance IDEs - Cool App Selection

News

Access to the CodeRage 2019 Playlists

News

Week of 2nd of December 2019

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.

IN THE ARTICLES