This feeder creates a stream starting from a web page. It is possible to define how often the page should be downloaded and parsed.
Every time the page is parsed a Message is sent down the lane.


urlSTRINGemptyURL of the web page
freqDURATION60show often the page should be parsed
text_onlyBOOL“false”if “true” it removes all the tags from the page
methodSTRING“GET”HTTP method to use on the requests
headersJSONemptyHeaders to use in the request
dataJSONemptyPOST fields to send with the requests (it’s not possible to use in combination with rawData)
rawDataSTRINGemptyraw body of the requests (it’s not possible to use in combination with data)
statusSTRINGemptythe filter will propagate the Message only if the returned status is this
cookiesSTRINGemptyPath of the JSON file containing the cookies to use
... | <web: url="https://example.com", freq="30m", status="200", cookies="/path/to/exported.json"> | ...



The main field of the Message will contain the HTML source or the text of the website if the text_only parameter is set to true.


urlURL of the web page
titlemeta tag title
descriptionmeta tag description
imagemeta tag image
sitenamemeta tag sitename
Not all the Extra field could be filled. If the relative tag is not present on the feed it will be empty.