This commit adds a new bridge for http://www.instructables.com. This bridge
currently supports fetching content by category (all categories available 200+),
using available filters (featured, recent, popular, views, contest winners).
Adds duration limits (minimum duration, maximum duration) for all
modes (user/id/playlist/search). Duration limits are optional, so
existing subscriptions don't break.
The limits are specified by two separate parameters, each of which
is optional:
- `&duration_min=` (minimum duration in minutes, default: -1)
- `&duration_max=` (maximum duration in minutes, default: INF)
If duration limits are specified in either user, id or playlist mode,
the bridge defaults to fetching data from HTML intead of XML feeds,
which requires more bandwidth and takes longer, because each video is
loaded individually!
References #670
The previous context is now labeled 'User', while the new context is
labeled 'Group'. The existing code was not changed, instead new group*
functions were implemented to handle groups.
The general principle of capturing groups is the same as done for users
with adjustments to account for different HTML structures.
Captcha responses are currently not supported for groups! There doesn't
seem to be a way to trigger them consistently, which makes it hard to
handle them properly.
Features of the group context:
- The feed title is based on the group name
- The group URI used for capturing is returned for the feed URI
- Author names and timestamps are reproduced from the source
- Post titles are reproduced from the source if they exist, otherwise
the title is build manually from the author name and the content
- Original contents are included with the feed
- All images are attached as enclosures as well
Closes #
Supports latest news and profiling a given ETF in Englisch, German
or Italian language. Cover images are attached as enclosures and not
as part of the content.
News:
Optionally loads the full article for each news item. Some articles
may include scripts to provide interactive graphs. These scripts are
removed as they would be rendered as pure text and a message is shown
instead: "[Content removed! Visit site to see full contents!]"
Profile:
Optionally includes the ETF strategy and description.
isoHunt has discontinued services due to legal reasons and is now
accessible via https://isohunts.to
While it is certainly possible to rewrite the bridge to fetch some
information from the new site, it wouldn't be able to provide as
much functionality as before. This is due to isoHunt having removed
all searching and filtering options, only providing static HTML pages
for general categories (anime, movies, etc...). Those pages, however,
are heavily broken.
Unless someone is interested in monitoring the general categories
the effort of upgrading the bridge to the new site is not worth taking
time for.
Users of isoHunt are asked to make use of their client application,
as they don't provide online services anymore (it's now in the darknet)
Here is the statement from isoHunt:
"Due to hard regulations and security issues for bittorrent users, we
have moved into a more secure and even faster district of the internet!
[...]
Torrent Downloads have a high risk of getting legal problems. That is
why we do not offer torrentfiles any more. [...]"
-- source: https://isohunts.to
Allows users to paste facebook links as user name. The link must contain
the correct host (www.facebook.com) and a valid path (/user-name/...).
The first part of the path is used for the user name. Errors are returned
in case something went wrong.
References #706
Reviews are provided the same way as summary posts and therefore returned
as separate feed item for each review. This commit adds a new option
'&skip_reviews=on' to skip reviews entirely.
References #706
The media_type parameter was recently replaced by media_type_u (for
user mode) and media_type_h (for hashtag mode). This was necessary
in order to add the media type 'story' only for the user mode.
"The reason for that is that RSS-Bridge supports multiple parameters
with the same name if and only if they contain the exact same value.
Here, hashtags don't have stories, so it would not be possible to
pass "story" as a parameter. This is a design mistake that I made
when I added support for hashtags."
-- 8770c87389 (r28871502)
However as pointed out this change breaks existing feeds as the
parameter name is no longer compatible to previous implementations.
This commit changes the implementation to provide the old media_type
parameter globally and check for invalid options on each request. If
a user uses the 'story' option in history mode the bridge returns a
client error.
references 8770c87
references #694fixes#696fixes#699fixes#701
* [DealabsBridge] Follow site changes, fix unhandled case
- Fixed the case where no discount was shown
- Changed some CSS class to follow the website changes
Now, entry title is optionnal and may be found in h3 HTML element.
Entry content is mandatory and may be found in div[class="item-content"] HTML element.
Moreover, the title may contain simple quotes (here, encoded) so the bridge have to decode first to apply format library function. In case we don't do that, the format function double encode the quote and something like ' could appear.
* [VkBridge] Convert special HTML entities to characters in pageName
* [VkBridge] Generate feed item title
* [VkBridge] Remove double backslashes in feed item link
* [VkBridge] Unpin post if pinned
* [VkBridge] Mark reposted messages
* [VkBridge] Correct external link parsing
* [VkBridge] Added article parsing
* [VkBridge] Added video parsing
* [VkBridge] Added photo parsing
* [VkBridge] Added album link parsing
* [VkBridge] Added one more external link selector
* [VkBridge] Using array of link selectors to remove
* [VkBridge] Added document parsing
* [VkBridge] Added sign parsing
* [VkBridge] Fixed incorrect sorting with pinned item
* [VkBridge] More methods to parse documents
* [VkBridge] Save fallback if page name element not found
* [VkBridge] Using post signed as feed item author
* [VkBridge] Fixed document link
* [VkBridge] Coding policy fixes
NotAlways right found it necessary to remove their RSS feeds recently. This is a *simple* bridge to grab the ones on the front page. It allows you to filter the articles based on their classification (right, working, romantic, related, learning, friendly, hopeless, unfiltered, or all).
The limit was used to specify the number of pages to return from a given
topic which resulted in the number of returned items variing between one
and however many entries are listed on one page.
This commit changes the implementation for the limit to keep loading more
pages until the specified limit is reached. Excessive elements are removed
in order to return the exact amount of items specified by the limit.
This behavior is closer to how other bridges are implemented and makes it
more natural to use without being too confusing. Existing queries must be
updated to account for the new limit.
References #657
Requesting a username with a leading slash would cause error 500
because the requested URI would contain two slashes in a row.
For example username "/test" would result in:
https://facebook.com//test
References #628
The author name is parsed by searching a string within the entire
HTML document:
$author = $html->innertext;
$author = substr($author, strpos($author, '"author=') + 8);
$author = substr($author, 0, strpos($author, '\u0026'));
This solution will return big portions of the HTML document if
the strpos function returns zero (not found).
This commit replaces the previous implementation by searching for
a specific script tag and making use of the JSON data inside it.
References #580
This bridge returns feeds for any URI that is compatible with the
IPB implementation (currently 4.x). Older versions might work, but
there is no guarantee.
Only forum and topic URIs are supported!
The bridge automatically checks if natural feeds are available (by
adding '.xml' to the URI). If so the feed is returned. Otherwise
the bridge will attempt to identify the content type and build a
feed accordingly.
Valid URIs are forums and topics. For forums the first page is
returned, for topics the last one. Elements are ordered such that
the latest entry is returned first (oldest-to-newest)
The optional parameter '&limit=' specifies how many pages should
be loaded (default: 1). Topics are loaded in reverse order.
=> Does not work with forums!
Images are provided as enclosures and scaled to a max-size of
400x400 pixels by default (Except for natural feeds).
The content is filtered before being returned:
- Unnecessary tags are removed (iframes, etc...)
- Styles for blockquotes are restored (grey background)
Closes#507
