Commit graph

2427 commits

Author SHA1 Message Date
LogMANOriginal
b90bcee1fc
Return exceptions in requested feed formats (#841)
* [Exceptions] Don't return header for bridge exceptions
* [Exceptions] Add link to list in exception message

This is an alternative when the button is not rendered
for some reason.

* [index] Don't return bridge exception for formats
* [index] Return feed item for bridge exceptions
* [BridgeAbstract] Rename 'getCacheTime' to 'getModifiedTime'
* [BridgeAbstract] Move caching to index.php to separate concerns

index.php needs more control over caching behavior in order to cache
exceptions. This cannot be done in a bridge, as the bridge might be
broken, thus preventing caching from working.

This also (and more importantly) separates concerns. The bridge should
not even care if caching is involved or not. Its purpose is to collect
and provide data.

Response times should be faster, as more complex bridge functions like
'setDatas' (evaluates all input parameters to predict the current
context) and 'collectData' (collects data from sites) can be skipped
entirely.

Notice: In its current form, index.php takes care of caching. This
could, however, be moved into a separate class (i.e. CacheAbstract)
in order to make implementation details cache specific.

* [index] Add '_error_time' parameter to $item['uri']

This ensures that error messages are recognized by feed readers as
new errors after 24 hours. During that time the same item is returned
no matter how often the cache is cleared.

References https://github.com/RSS-Bridge/rss-bridge/issues/814#issuecomment-420876162

* [index] Include '_error_time' in the title for errors

This prevents feed readers from "updating" feeds based on the title

* [index] Handle "HTTP_IF_MODIFIED_SINCE" client requests

Implementation is based on `BridgeAbstract::dieIfNotModified()`,
introduced in 422c125d8e and
simplified based on https://stackoverflow.com/a/10847262

Basically, before returning cached data we check if the client send
the "HTTP_IF_MODIFIED_SINCE" header. If the modification time is
more recent or equal to the cache time, we reply with "HTTP/1.1 304
 Not Modified" (same as before). Otherwise send the cached data.

* [index] Don't encode exception message with `htmlspecialchars`
* [Exceptions] Include error message in exception
* [index] Show different error message for error code 0
2018-10-15 17:21:43 +02:00
logmanoriginal
996295e82f Add 'dev.' to the release version in master
This helps (roughly) identifying versions when opening issues on
GitHub, using the latest ZIP file for master.

References #773
2018-09-26 20:04:27 +02:00
logmanoriginal
13bd7fe21b [contents] Return error if the server responded with any code other than 200 2018-09-26 19:16:02 +02:00
logmanoriginal
fcc9f9fd61 [FacebookBridge] Use alternative URI to load more posts
The URI "https://facebook.com/username?_fb_noscript=1" returns two
posts per user. Some profiles, however, are very active, causing the
bridge to miss items if more than two posts are send within the cache
duration (5 minutes).

The alternative suggested in #669 is to use a different URI:
"https://facebook.com/pg/username/posts?_fb_noscript=1"

While the contents of this URI essentially look the same when viewed
in a browser, it actually returns more than 10 posts depending on the
profile.

References #669
2018-09-26 18:24:46 +02:00
logmanoriginal
e1c4914b1c [FacebookBridge] Optimize for readability 2018-09-25 18:56:33 +02:00
logmanoriginal
93e7ea9fea [HtmlFormat] Make feeds available via syndication links 2018-09-22 19:51:18 +02:00
logmanoriginal
2d1b446bd1 [DevToBridge] Add new bridge
Returns feeds for tags from https://dev.to

References #840
2018-09-22 18:57:07 +02:00
logmanoriginal
1d451610d6 [ParameterValidator] Move 'getQueriedContext' from BridgeAbstract 2018-09-22 17:04:55 +02:00
logmanoriginal
f853ffc07c [ParameterValidator] Refactor 'validation' into 'ParameterValidator'
Adds a new class 'ParameterValidator' to replace the functions from
'validator.php', separating private functions from 'validateData' to
class private functions in the process.

Instead of echoing error messages, adds messages to a private variable,
accessible via 'getInvalidParameters'.

BridgeAbstract now adds invalid parameter names to the error message.
2018-09-22 16:42:04 +02:00
logmanoriginal
e3a5a6a170 [index] Update and improve parameter handling for bridge and cache
- Use 'array_diff_key' instead of 'unset'
- Remove parameters for caches

By removing certain parameters for caches, the loading times can be
improved considerably:

* action: It doesn't matter which action the user took to generate
feed items.

* format: This has the biggest impact on performance, because cached
items are now shared between different formats (i.e. try switching
between Atom, Html and Mrss and compare previous vs. now). If a
server handles lots of requests, this may even reduce bandwidth if
the same contents are requested for different formats.

* _noproxy: The proxy behavior has no impact on the produced items,
so it can be ignored.

* _cache_timeout: This is another option which might impact performance
for some servers, especially if 'custom_timeout' has been enabled in
the configuration. Requests with different cache timeouts no longer
result in separate cache files.
2018-09-22 15:44:03 +02:00
logmanoriginal
243e324efc [NineGagBridge] Fix missing sections breaking feeds
Posts may supply a list of 'sections' or a single 'postSection'

References #844
2018-09-22 15:19:14 +02:00
logmanoriginal
ae58b1566e [NineGagBridge] Remove type hinting
Type hinting for strings doesn't work prior to PHP 7, see
http://php.net/manual/en/functions.arguments.php#functions.arguments.type-declaration

References #837
2018-09-22 15:19:14 +02:00
sysadminstory
c044694b21 [ZoneTelechargementBridge] Sort episodes from newest to oldest (#835)
References #834
2018-09-21 20:22:49 +02:00
triatic
db24f55c86 [FB2Bridge] Do not strip <h3> and <h4> (#836)
Do not strip <h3> and <h4>. Output looks better when they are retained. See attached.
2018-09-21 20:19:22 +02:00
logmanoriginal
eb30038d6b [README] Update and reorganize 2018-09-16 18:20:35 +02:00
logmanoriginal
712a581ed6 [README] Add badge for Guix release
Unfortunately there is no way to query the current package version,
so this is only a placeholder
2018-09-16 16:01:51 +02:00
logmanoriginal
d3df4b51b8 [README] Add badge for current debian release 2018-09-16 15:13:30 +02:00
logmanoriginal
e6476a600d [KununuBridge] Fix broken bridge and simplify implementation 2018-09-16 09:55:35 +02:00
Grégory T
811e8d8c88 [ETTVBridge] Improvements and bug fixes (#682)
* Fix typo with status field
* Comply with other bridges

Change the uri element of an item to point, not on the magnet link, but on the page, as similar bridges do.

* Improved to return name & uri matching with query

This change makes it possible for the feed reader to discover a title and url consistent with the user's search.
2018-09-15 17:11:36 +02:00
logmanoriginal
adc6f72e97 [style] Fix first letter of labels not capitalized
This error is caused by setting label::before { content: " "; },
which makes the first letter a whitespace on all labels, neccessary
 for browsers that doesn't support the grid layout.

This commit clears the content if the browser supports the grid layout,
properly capitalizing labels again. If a browser doesn't support grid
layout, labels stay as they are provided by the bridge.
2018-09-15 17:04:20 +02:00
logmanoriginal
182153485c [Arte7Bridge] Move parameter examples into tool tip for readability 2018-09-15 16:50:10 +02:00
LogMANOriginal
bf9946d1fc
CSS adjustments to improve readability for bridge parameters (#763)
* Group common selectors
* Fix indentation using tabs
* Use same styles for number and text inputs
* Use grid layout for parameters

Introduces the grid layout for bridge parameters. All parameters are
arranged in a grid to improve readability. Read more on grid layouts
at

- https://www.w3schools.com/css/css_grid.asp
- https://developer.mozilla.org/en-US/docs/Web/CSS/CSS_Grid_Layout

Notice:

Grid layouts are not supported in very old browser versions:
https://developer.mozilla.org/en-US/docs/Web/CSS/CSS_Grid_Layout/CSS_Grid_and_Progressive_Enhancement

This is why @supports checks for browser support (not supported in IE)
https://developer.mozilla.org/en-US/docs/Web/CSS/@supports#Browser_compatibility

In case grid layout is not supported, the displayed form is usable
but not very pretty due to <br> being removed by this commit for
cosmetic reasons (breaks grid layout).

Unfortunately it doesn't seem possible to insert line breaks manually
via '::after { content: '\A' }' in cases where grid layout isn't
supported.

* Add padding to card parameters

Adds padding to parameters to improve readability. For bridges without
parameters (count($parameters) === 0), the parameter 'div' is no longer
created.

* Add colon ':' after label via CSS
* Capitalize first letter of label for readability
* Fix checkbox isn't aligned left

Sets the size of the checkbox to 20x20 px for good measure.

* Harmonize formatting
* Add new style to number and select boxes

References #797

* Add fallback solution for browsers without grid support
2018-09-15 16:39:50 +02:00
triatic
ec60752650 [FB2Bridge] Prevent Facebook link href's ending in two quotes (#831)
Additionally prevent Facebook links having two forward slashes after the hostname.
2018-09-15 15:16:15 +02:00
sysadminstory
6688cf0c3b [AutoJMBridge] Fix concatenation bug (#833) 2018-09-15 15:12:34 +02:00
ORelio
ae45a8cfee [contents] Fix open_basedir warning (#832)
References #818
2018-09-15 14:46:11 +02:00
Matthew Seal
e34ef6cb4f [MrssFormat] Escape double quotes in XML attributes (#813)
XML attributes need to have certain characters escaped to be valid. The title attribute can have double quotes in it which need to be properly encoded for attributes.
2018-09-15 14:13:05 +02:00
sysadminstory
5c92a736fa [ZoneTelechargementBridge] Added Bridge for ww2.zone-telechargement1.org (#829)
* [ZoneTelechargementBridge] Added Bridge for ww2.zone-telechargement1.org

Goal for this bridge is to follow the episode publication of a TV show
season while it's broadcasted on the TV.
2018-09-13 19:36:48 +01:00
Eugene Molotov
911bcfb246 [PikabuBridge] Implemented bridge (#830)
* [PikabuBridge] Implemented bridge
2018-09-13 12:52:26 +01:00
ZeNairolf
efa550ef61 Add 9gag.com bridge (#801)
* Add 9gag.com bridge
2018-09-13 10:11:42 +01:00
sysadminstory
d5d7683ed3 [AutoJMBridge] New Bridge (#827)
* [AutoJMBridge] New Bridge

This bridge will show all the car offers AutoJM has for the model you
choosed and using your filter. Very useful to wait for a cheap price for
a new car !
2018-09-13 10:05:07 +01:00
triatic
fe94914eb5 [AtomFormat.php] Eliminate PHP Notice when running in CLI mode (#824) 2018-09-12 14:37:27 +01:00
Quentin Delmas
622802e5d4 Fix multiple warnings.
Fix JSON request string in case of empty location
2018-09-12 13:31:11 +01:00
sysadminstory
6da8daf1a3 [DealabsBridge] Fix for #782 and all categories are now available (#821)
This commit fixes #782 by updating the parameter value of 'Maison &
Jardin', but this means the user has to update his RSS Feed URL (.because
of the bridge structure, it would be a nightmare to fix it in another
way)

This commits add all the categories available on Dealabs Website.
2018-09-11 22:11:00 +01:00
la Bécasse
654e502e84 Arte7 collection support (#819)
* Arte7 collection support
2018-09-11 22:09:47 +01:00
sysadminstory
c8ace9e3bd [ExtremeDownloadBridge] Added Bridge for ww1.extreme-d0wn.com (#820)
* [ExtremeDownloadBridge] Added Bridge for ww1.extreme-d0wn.com

Goal for this bridge is to follow the episode publication of a TV show season
while it's broadcasted on the TV.
2018-09-11 20:10:46 +01:00
Monsieur Poutounours
5722a6c139 Adding a bridge for theyetee.com (#809)
* Adding a bridge for theyetee.com

The bridge fetches daily shirts at theyetee.com.
The Yetee offers two new shirts each day, but you can buy them only for a few hours !
Unfortunately, the site don't provide RSS feed, so the only way to keep up to date on new shirt is their daily mailing ... until now !
2018-09-10 20:56:55 +01:00
Quentin Delmas
458b826871 Remove declaration of extractFromDelimiters, it is now a reusable function. Fixes #815 2018-09-10 09:29:19 +01:00
Quentin Delmas
b397a42876 version: Bump to 2018-09-09 2018-09-09 21:00:10 +01:00
Corentin Garcia
111c45d010 [GithubSearchBridge] Fix content parsing, add tags if present (#803)
* [GithubSearchBridge] Fix content parsing, add tags if present

* [GithubSearchBridge] Add categories (from tags)
2018-09-09 20:30:29 +01:00
Corentin Garcia
55b36b0455 [DauphineLibereBridge] Use https, fix content parsing (fix issue #780) (#811) 2018-09-09 20:23:59 +01:00
ORelio
de8cee6a1c Catching up | [Main] Debug mode, parse utils, MIME | [Bridges] Add/Improve 20 bridges (#802)
* Debug mode improvements

 - Improve debug warning message
 - Restore error reporting in debug mode
 - Fix 'notice' messages for unset fields

* Add parsing utility functions

html.php
 - extractFromDelimiters
 - stripWithDelimiters
 - stripRecursiveHTMLSection
 - markdownToHtml (partial)

bridges
 - remove now-duplicate functions
 - call functions from html.php instead

* [Anidex] New bridge

Anime torrent tracker

* [Anime-Ultime] Restore thumbnail

* [CNET] Recreate bridge

Full rewrite as the previous one was broken

* [Dilbert] Minor URI fix

Use new self::URI property

* [EstCeQuonMetEnProd] Fix content extraction

Bridge was broken

* [Facebook] Fix "SpSonsSoriSsés" label

... which was taking space in item title

* [Futura-Sciences] Use HTTPS, More cleanup

Use HTTPS as FS now offer HTTPS
Clean additional useless HTML elements

* [GBATemp] Multiple fixes

- Fix categories: missing "break" statements
- Restore thumbnail as enclosure
- Fix date extraction
- Fix user blog post extraction
- Use getSimpleHTMLDOMCached

* [JapanExpo] Fix bridge, HTTPS, thumbnails

- Fix getSimpleHTMLDOMCached call
- Upgrade to HTTPS as JE now offers HTTPS
- Restore thumbnails as enclosures

* [LeMondeInformatique] Fix bridge, HTTPS

- Upgrade to HTTPS as LMI now offers HTTPS
- Restore thumbnails using small images
- Fix content extraction
- Fix text encoding issue

* [Nextgov] Fix content extraction

- Restore thumbnail and use small image
- Field extraction fixes

* [NextInpact] Add categories and filtering by type

- Offer all RSS feeds
- Allow filtering by article type
- Implement extraction for brief articles
- Remove article limit, many brief articles are publied all at once

* [NyaaTorrents] New bridge

Anime torrent tracker

* [Releases3DS] Cache content, restore thumbnail

- Use getSimpleHTMLDOMCached
- Restore thumbnail as enclosure

* [TheHackerNews] Fix bridge

 - Fix content extraction including article body
 - Restore thumbnail as enclosure

* [WeLiveSecurity] HTTPS, Fix content extraction

- Upgrade to HTTPS as WLS now offers HTTPS
- Fix content extraction including article body

* [WordPress] Reduce timeout, more content selectors

- Reduce timeout to use default one (1h)
- Add new content selector (articleBody)
- Find thumbnail and set as enclosure
- Fix <script> cleanup

* [YGGTorrent] Increase limit, use cache

- Increase item limit as uploads are very frequent
- Use getSimpleHTMLDOMCached

* [ZDNet] Rewrite with FeedExpander

- Upgrade to HTTPS as ZD now offers HTTPS
- Use FeedExpander for secondary fields
- Fix content extraction for article body

* [Main] Handle MIME type for enclosures

Many feed readers will ignore enclosures (e.g. thumbnails) with no MIME type. This commit adds automatic MIME type detection based on file extension (which may be inaccurate but is the only way without fetching the content).

One can force enclosure type using #.ext anchor (hacky, needs improving)

* [FeedExpander] Improve field extraction

- Add support for passing enclosures
- Improve author and uri extraction
- Fix 'notice' PHP error messages

* [Pull] Coding style fixes for #802

* [Pull] Implementing changes for #802

 - Fix coding style issues with str append
 - Remove useless CACHE_TIMEOUT
 - Use count() instead of $limit
 - Use defaultLinkTo() + handle strings
 - Use http_build_query()
 - Fix missing </em>
 - Remove error_reporting(0)
 - warning CSS (@LogMANOriginal)
 - Fix typo in FeedExpander comment

* [Main] More documentation for markdownToHtml

See #802 for more details
2018-09-09 20:20:13 +01:00
Quentin Delmas
123fce4394 [ForGifsBridge] Fix permissions of ForGifsBridge 2018-09-09 17:34:36 +01:00
Quentin Delmas
a3f99c9c3f [GOGBridge] Added bridge for GOG.com 2018-09-09 17:32:36 +01:00
Eugene Molotov
bf30ad127c [FacebookBridge] Removes query string from post links
* [FacebookBridge] Removes query string from post links
2018-09-09 16:31:15 +01:00
logmanoriginal
37f84196b7 [GooglePlusPostBridge] Fix title is empty if content is too short
The bridge would generate empty titles if the content is longer than
50 characters, but doesn't have further spaces in it. With this commit
the title is correctly generated based on the contents, taking missing
spaces into account.

References #786
2018-09-08 17:07:57 +02:00
Corentin Garcia
44764f7182 [GrandComicsDatabaseBridge] Fix links in content (#804) 2018-09-08 11:12:27 +01:00
Antoine Cadoret
19f294d71d Add fields to leboncoin bridge (#783)
* [LeBonCoinBridge] Add fields to LeBonCoinBridge
2018-08-31 14:34:41 +01:00
Teromene
b0e33e4e01
Update LeBonCoinBridge to use the site's API (#795)
* Update LeBonCoinBridge to use the site's API
2018-08-28 14:20:02 +01:00
Eugene Molotov
558fa50a2a [core] Enabled debug mode before including core files (#790) 2018-08-25 20:02:47 +01:00
Eugene Molotov
ffb8b82c73 [FileCache] reseting cached file stat result to have correct getTime() result (#792)
* [FileCache] reseting cached file stat result to have correct getTime() result
2018-08-25 20:00:51 +01:00