Need to extract all the data from a webpage

Discussion Ask a question
,
1

So, i was trying to extract and get all the data (news) and get a json format of that data from a particular webpage. So searched google and found its called ‘Data Scraping’. I tried to do this, but it doesn’t give any result. Any solution plz ?

URL from where i want to extract the data : https://www.paladins.com/news

This is the backend code…

1920×953 33 KB

This is the frontend code…

1919×950 27.4 KB

Result i got…

1919×950 20.2 KB

Have i done wrong on the code part…if yes…plz help me correct it !!!

2

Hello

To do this you would need to send a fetch request to the api part of that website, get the result from the fetch request and then insert it into a collection in your backend. It would look something like this:

import {fetch} from 'wix-fetch';

// ...

fetch("  https://www.paladins.com/news", {method: "get"})
  .then( (httpResponse) => {
    if (httpResponse.ok) {
      return httpResponse.json();
    } else {
      return Promise.reject("Fetch did not succeed");
    }
  } )
  .then(json) => {
  let toInsert = {
  "title":        json.name,
  "first_name":   json.fname,
  "last_name":    json.lname
};

wixData.insert("myCollection", toInsert)
  .then( (results) => {
		let item = results; //see item below
	} )
	.catch( (err) => {
		let errorMsg = err;
	} );
   })
  .catch(err => console.log(err));

Good luck!
Majd

3

Thanks for the reply. I did what u said , but its showing error " Invalid JSON response body ". Can u plz tell me step by step.

4

How would I proceed with this function if the website has no API? I purchased a subscription for the data but no API is offered and I would like to extract the data as a search function on my wix site.

5

This post can help you to understand your situation.
https://www.wix.com/velo/forum/coding-with-velo/how-can-i-use-json-object-data-to-populate-a-text-element-in-response-to-user-action