The ScrapeStack API was built to offer a simple REST API interface for scraping web pages at scale without having to programmatically deal with geolocations, IP blocks or CAPTCHAs. The API supports a series of features essential to web scraping, such as JavaScript rendering, custom HTTP headers, various geo-targets, POST/PUT requests and an option to use premium residential proxies instead of datacenter proxies.
In this article, we will see how fast and easy it is to use Windows App Development tools like RAD Studio and Delphi to create a Firemonkey multidevide application using the LowCode Wizard in addition to a REST client library to take advantage of ScrapeStack API and retrieve a JSON format response for Website Scrape in real-time.
Our RAD Studio and Delphi applications will be able to call the API and request information based on the name of parameters you provide
Table of Contents
How do I set up the ScrapeStack API?
Make sure you refer to ScrapeStack API website – https://ScrapeStack.com/ – and SignUp for the free Plan providing only your email and some basic information (no credit card required). Once you are in the website will redirect you to a Quickstart guide dashboard and your API Access Key will be provided. The Access Key unique, personal and is required to authenticate with the API. Keep it safe!
How do I call ScrapeStack API endpoints?
Now all we need to do is to call the API base URL http://api.scrapeStack.com/ via a HTTP POST method with no JSON request body needed and some few requested parameters added to the URL address depending on the endpoint we choose to call. One can do that using REST Client libraries available on several programming languages.
scrapeStack offers a single endpoint to scrape whatever webpage you want. For a complete and detailed information on endpoint and its parameters make sure you refer to ScrapeStack Quickstart guide at https://scrapeStack.com/quickstart
|
1 2 3 4 5 6 7 8 9 10 11 |
// Scrape web page by providing a URL http://api.scrapestack.com/scrape ? access_key = YOUR_ACCESS_KEY & url = https://apple.com // optional parameters: & render_js = 0 & keep_headers = 0 & proxy_location = us & premium_proxy = 0 // more parameters available please refer to the API Documentation (<a href="https://scrapeStack.com/documentation" target="_blank" rel="noreferrer noopener nofollow">https://scrapeStack.com/documentation</a>) |
What does the ScrapeStack API endpoint return?
After the call the main results will be the complete scrape of the website that you provided in the parameter URL. In this case we provided Embarcadero’s website https://www.embarcadero.com – part of the result is shown in the screenshot below.
How do I connect my Delphi applications to ScrapeStack API?
Once you have followed basic steps to set up the ScrapeStack API we need to make sure we are able to connect and communicate with it before we start writing some code.
RAD Studio Delphi and C++Builder make it very easy to connect to APIs as you can you REST Debugger to automatically create the REST components and paste them into your app.
In Delphi all the job is done using 3 components tot make the API call. They are the TRESTClient, TRESTRequest, and TRESTResponse. Once you connect the REST Debugger successfully, copy and past the components you will notice that the API URL is set on the BaseURL of TRESTClient. On the TRESTRequest component you will see that the request type is set to rmPOST, the ContentType is set to ctAPPLICATION_JSON, and that it contains one request body for the POST.
Run your RAD Studio Delphi and on the main menu click on Tools > REST Debbuger. Configure the REST Debugger as follows marking the content-type as application/json, and adding the POST url, the JSON request body and the API key you created. Once you click the Send Request button you should see the JSON response, just like we demonstrated below.
How do I build a Windows, MacOS desktop or Android/iOS mobile app using the ScrapeStack API?
Low code app development is increasingly popular today, and for good reason: it speaks to the demand for software paired with the difficulty of creating it. In many ways, Delphi led the low code revolution: RAD, where you build software by connecting components in a visual designer, is low code. The key with many low code solutions is extensibility: low code which cannot be extended to a traditionally coded app has a hard upper limit of functionality. If you’re a technology adviser, you do not want to recommend something which will limit your client or company. The best development solutions that target low code provide an avenue to grow and extend. In this too, Delphi excels.
Until recently in Delphi you still had to create a new project and create the forms or screens yourself – no matter how little code you may have to write to get powerful functionality when you did. This is something we’ve now addressed with an addon to 10.4.2. Run your RAD Studio Delphi and on the main menu click on Tools > GetIt Package Manager. In GetIt for 10.4.2 and later search for “Low Code App Wizard for FireMonkey”, our low code technology for cross-platform applications.
How to configure API REST calls in Delphi
Now that you have installed Low Code App Wizard for FireMonkey and that you were able to successfully configure and test your API calls on the REST Debugger, go back to the REST Debugger and click the Copy Components button. Open RAD Studio Delphi and on the main menu click File > New> Multi-Device Application Delphi, select the FireMonkey Template App as shown below and follow some simples Wizard steps to create your Low Code application.
The result will be a fully functional cross-platform application, which already contains basic functionality that every mobile app will need — you can run and use the app as is — and hooks to build more, plus demonstrates application development best practices with a very clean extensible design to add more screens with low code, plus a set of tests to ensure your app remains high quality as you customise it.
Are apps built with the RAD Studio Low Code Wizard configurable?
The app that Delphi builds for you is easily configurable through the wizard and you can choose any set of the following:
- User accounts: sign up, sign in, and a profile screen, with hooks for your own authentication
- Settings screen (where you can change the app theme (light or dark), plus some example settings you can reuse or extend)
- Contacts screen
- Terms of Use, About, and Privacy policy screens, easily configurable for your own data
- A database layer: using Interbase, store data in your app
- An example new screen you can use as a template or example for adding new functionality
As you can see the Wizard saves you a lot of time as it creates a complete project with many forms ready for you to implement the code. Open the recently created NewFormFrame and hit Ctrl+V in order to paste the components you copied from REST Debugger. This action will result in three components added to the the form RESTClient1, RESTRequest1 and RESTResponse1.
How do I add custom code to a Delphi app created with the RAD Studio Low Code app wizard?
Now we will simply add very simple code to a TButton OnClick event to make sure every thing is configured correctly and voila! In five minutes we have made our very first call to ScrapeStack API and we are able to receive JSON response for whatever parameter we want.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 |
procedure TNewFormFrame.Button1Click(Sender: TObject); begin inherited; memo1.Lines.text := ''; RESTClient1.ResetToDefaults; RESTClient1.Accept := 'application/json, text/plain; q=0.9, text/html;q=0.8,'; RESTClient1.AcceptCharset := 'UTF-8, *;q=0.8'; RESTClient1.BaseURL := 'http://api.scrapestack.com/scrape'; RESTClient1.HandleRedirects := True; RESTClient1.RaiseExceptionOn500 := False; //here is were we pass the access_key and the additional parameters RESTRequest1.Resource := Format('?access_key=%s&url=%s', [edit1.Text, edit2.Text]); RESTRequest1.Client := RESTClient1; RESTRequest1.Response := RESTResponse1; RESTRequest1.SynchronizedEvents := False; RESTResponse1.ContentType := 'application/json'; RESTRequest1.Execute; memo1.Lines.text := RESTResponse1.Content; end; |
The sample application features a TEdit as a place to paste in the access key and additional TEdit for each parameter, a TMemo to display the JSON results of the REST API call. Now you have every thing you need in order to interate with the response data and make your application process the information and present it visually in the way it better suits your needs!
Screenshot of the final Delphi example web scraper app example running
In this blog post we’ve seen how to sign up for the ScrapeStack API in order to get instant access to structured and readable website scrape data. We’ve seen how to use the RAD Studio REST Debugger to connect to the endpoint and copy that code into a real application. And finally we’ve seen how easy and fast it is to use RAD Studio Delphi and the Low Code App Wizard for FireMonkey to create a real Multidevice (Windows/MacOS desktop or Android/iOS mobile) application which connects to the API and retrieves the data ready for you to iterate!
You can download the full Delphi source code for the website-scraping demo application here: https://github.com/checkdigits/ScrapestackAppAPI_example
Did you know you can download a FREE trial of RAD Studio Delphi and try out this example for yourself? Click here to try RAD Studio and see how it truly is Rapid Application Development.
Design. Code. Compile. Deploy.
Start Free Trial Upgrade Today
Free Delphi Community Edition Free C++Builder Community Edition
