Commit graph

1656 commits

Author SHA1 Message Date
logmanoriginal 1cb83ccea3 [IPBBridge] Use limit for the number of items
The limit was used to specify the number of pages to return from a given
topic which resulted in the number of returned items variing between one
and however many entries are listed on one page.

This commit changes the implementation for the limit to keep loading more
pages until the specified limit is reached. Excessive elements are removed
in order to return the exact amount of items specified by the limit.

This behavior is closer to how other bridges are implemented and makes it
more natural to use without being too confusing. Existing queries must be
updated to account for the new limit.

References #657
2018-04-06 22:25:49 +02:00
sysadminstory c899399569 [DealabsBridge] Follow the website changes (#660) 2018-04-06 21:25:41 +02:00
LogMANOriginal 0f93370e92
Merge pull request #654 from LogMANOriginal/cURL
Use cURL instead of file_get_contents
2018-04-06 20:49:58 +02:00
logmanoriginal 45c3dcb636 [VkBridge] Simplify header specification 2018-04-06 20:42:19 +02:00
logmanoriginal ecfc220b10 [KernelBugTrackerBridge] Fix too many parameters requesting HTML DOM 2018-04-06 20:42:19 +02:00
logmanoriginal 4b3efed7ec [YoutubeBridge] Fix too many parameters when using HTML mode 2018-04-06 20:42:19 +02:00
logmanoriginal bc28c5da8e [contents] Set CURLOPT_HTTPHEADER only if the provided array contains data 2018-04-06 20:42:19 +02:00
logmanoriginal 5bd9c1611d [contents] Limit cURL protocols to HTTP and HTTPS 2018-04-06 20:42:19 +02:00
logmanoriginal 6caca4946b bridges: Fix bridges with custom headers and options
This commit fixes bridges which called getContents, getSimpleHTMLDOM
or getSimpleHTMLDOMCached with custom settings.
2018-04-06 20:42:19 +02:00
logmanoriginal ee78e7613f [contents] Replace file_get_contents by cURL
cURL is a powerful library specifically designed to connect to many
different types of servers with different types of protocols. For
more detailed information refer to the PHP cURL manual:

- http://php.net/manual/en/book.curl.php

Due to this change some parameters for the getContents function were
necessary (also applies to getSimpleHTMLDOM and getSimpleHTMLDOMCached):

> $use_include_path removed

  This parameter has never been used and doesn't even make sense in
  this context; If set to true file_get_contents would also search
  for files in the include_path (specified in php.ini).

> $context replaced by $header and $opts

  The $context parameter allowed for customization of the request in
  order to change how file_get_contents would acquire the data (i.e.
  using POST instead of GET, sending custom header, etc...)

  cURL also provides facilities to specify custom headers and change
  how it communicates to severs. cURL, however, is much more advanced.

  - $header is an optional parameter (empty by default). It receives
    an array of strings to send in the HTTP request header.

    See 'CURLOPT_HTTPHEADER':

    "An array of HTTP header fields to set, in the format
    array('Content-type: text/plain', 'Content-length: 100')"

    - php.net/manual/en/function.curl-setopt.php

  - $opts is an optional parameter (empty by default). It receives
    an array of options, where each option is a key-value-pair of
    a cURL option (CURLOPT_*) and it's associated parameter. This
    parameter accepts any of the CURLOPT_* settings.

    Example (sending POST instead of GET):

    $opts = array(
      CURLOPT_POST => 1,
      CURLOPT_POSTFIELDS => '&action=none'
    );
    $html = getContents($url, array(), $opts);

    Refer to the cURL setopt manual for more information:
    - php.net/manual/en/function.curl-setopt.php

> $offset and $maxlen removed

  These options were supported by file_get_contents, but there doesn't
  seem to be an equivalent in cURL. Since no caller uses them they are
  safe to remove.

Compressed data / Encoding

  By using cURL instead of file_get_contents RSS-Bridge no longer has
  to handle compressed data manually.

  See 'CURLOPT_ENCODING':

  "[...] Supported encodings are "identity", "deflate", and "gzip".
  If an empty string, "", is set, a header containing all supported
  encoding types is sent."

  - http://php.net/manual/en/function.curl-setopt.php

  Notice: By default all encoding types are accepted (""). This can
  be changed by setting a custom option via $opts.

    Example:

    $opts = array(CURLOPT_ENCODING => 'gzip');
    $html = getContents($url, array(), $opts);

Proxy

The proxy implementation should still work, but there doesn't seem
to be an equivalent for 'request_fulluri = true'. To my understanding
this isn't an issue because cURL knows how to handle proxy communication.
2018-04-06 20:42:19 +02:00
logmanoriginal 2df2623430 [index] Add 'curl' extension check 2018-04-06 20:42:19 +02:00
logmanoriginal de5f850cdb [index] Fix indentation using tabs 2018-04-06 20:34:44 +02:00
teromene ac6847045c Catch Errors in order to display a message in more cases. We also catch Exceptions to maintain compat with php 5.
Add check for simplexml extension.
2018-04-04 19:02:40 +01:00
logmanoriginal df6da837dc [FacebookBridge] Return error if username starts with slash
Requesting a username with a leading slash would cause error 500
because the requested URI would contain two slashes in a row.

For example username "/test" would result in:
https://facebook.com//test

References #628
2018-03-23 21:23:30 +01:00
Eugene Molotov 41b7984a4e [YoutubeBridge] Playlist mode: faster feed generating if item count is less or equal to 15 (#648)
* [YoutubeBridge] Playlist mode: faster feed generating if item count is less or equal to 15
2018-03-19 12:41:52 +00:00
teromene 38c7e0272e Add hashtag support to InstagramBridge.
Fixes  #629
2018-03-19 12:29:24 +00:00
teromene 29c690dbcd Fix InstagramBridge, thanks to @pintassilgo comments.
Fixes #646
2018-03-19 12:17:42 +00:00
LogMANOriginal 8ba817478b
Implement customizable cache timeout (#641)
* [BridgeAbstract] Implement customizable cache timeout

The customizable cache timeout is used instead of the default cache
timeout (CACHE_TIMEOUT) if specified by the caller.

* [index] Add new global parameter '_cache_timeout'

The _cache_timeout parameter is an optional parameter that can be
used to specify a custom cache timeout. This option is enabled by
default.

It can be disabled using the named constant 'CUSTOM_CACHE_TIMEOUT'
which supports two states:

> true: Enabled (default)
> false: Disabled

* [BridgeAbstract] Change scope of 'getCacheTimeout' to public

* [html] Add cache timeout parameter to all bridges

The timeout parameter only shows if CUSTOM_CACHE_TIMEOUT has been set
to true. The default value is automatically set to the value specified
in the bridge.

* [index] Disable custom cache timeout by default
2018-03-14 18:06:36 +01:00
Eugene Molotov cacbe90102 [YoutubeBridge] Sort playlist items by publication date (#643) 2018-03-13 11:24:40 +00:00
Antoine Cadoret cb91cd5d2f Fix SteamBridge (#637) (#639)
Fixes #639
2018-03-12 09:22:34 +00:00
sysadminstory 52dfa3fe76 [RadioMelodieBridge] Add new bridge (#640) 2018-03-11 15:38:07 +01:00
logmanoriginal 29a1c7ac09 [index.php] Add extension check for 'mbstring'
The mbstring extension is required by all formats in order to convert multi-
byte characters to UTF-8. This commit adds an extension check to throw an
error message if the extension is not enabled.
2018-03-07 19:11:47 +01:00
teromene 6eea51eeeb Fix SteamBridge.
Fixes #636
2018-03-07 10:24:33 +00:00
teromene 2149af0e74 Fix Pinterest bridge, remove the old JSON parsing, and return original sized image.
Fixes #632
2018-03-06 12:01:48 +00:00
teromene 142a647b7a Merge branch 'master' of github.com:RSS-Bridge/rss-bridge 2018-03-06 11:27:37 +00:00
teromene 6e916ddd35 Fix Arte7Bridge.
Fixes #633
2018-03-06 11:26:16 +00:00
Eugene Molotov 159b00145d [VkBridge] Setting feed title (#635)
* [VkBridge] Setting feed title
2018-03-05 09:46:15 +00:00
Mitsukarenai 26ce16baa2 [PlanetLibre] remove bridge (origin now has RSS) 2018-03-03 21:04:40 +01:00
sysadminstory 0622fe142b Dealabs : Added Groupes Feeds and Feed name is set according to parameters (#630)
* [DealabsBride] Added Groupes Feeds
2018-03-01 17:10:34 +00:00
logmanoriginal 4805b52d42 [YoutubeBridge] Fix typo 2018-02-16 22:35:00 +01:00
logmanoriginal 962617086e [YoutubeBridge] Remove superfluous div selectors 2018-02-16 22:31:47 +01:00
logmanoriginal 4f6277b6b5 [YoutubeBridge] Fix parsing author name breaks the bridge
The author name is parsed by searching a string within the entire
HTML document:

$author = $html->innertext;
$author = substr($author, strpos($author, '"author=') + 8);
$author = substr($author, 0, strpos($author, '\u0026'));

This solution will return big portions of the HTML document if
the strpos function returns zero (not found).

This commit replaces the previous implementation by searching for
a specific script tag and making use of the JSON data inside it.

References #580
2018-02-16 22:31:29 +01:00
logmanoriginal 5aaab9eb8c [YoutubeBridge] Skip unavailable videos 2018-02-16 22:11:03 +01:00
sysadminstory ef402bb5c3 [DealabsBride] Fix for the new site (#595)
* [DealabsBride] Fix for the new site
2018-02-14 11:03:44 +00:00
LogMANOriginal 85ac9001d6
[IPBBridge] Add bridge (#564)
This bridge returns feeds for any URI that is compatible with the
IPB implementation (currently 4.x). Older versions might work, but
there is no guarantee.

Only forum and topic URIs are supported!

The bridge automatically checks if natural feeds are available (by
adding '.xml' to the URI). If so the feed is returned. Otherwise
the bridge will attempt to identify the content type and build a
feed accordingly.

Valid URIs are forums and topics. For forums the first page is
returned, for topics the last one. Elements are ordered such that
the latest entry is returned first (oldest-to-newest)

The optional parameter '&limit=' specifies how many pages should
be loaded (default: 1). Topics are loaded in reverse order.
=> Does not work with forums!

Images are provided as enclosures and scaled to a max-size of
400x400 pixels by default (Except for natural feeds).

The content is filtered before being returned:
- Unnecessary tags are removed (iframes, etc...)
- Styles for blockquotes are restored (grey background)

Closes #507
2018-02-13 21:46:33 +01:00
Mitsukarenai 7939bffcdd fix: TébéoBridge Travis cleanup 2018-02-11 19:08:19 +01:00
Mitsukarenai bb58aa8e31 New bridge: Tébéo 2018-02-11 16:56:34 +01:00
Ruslan 1d35149191 Update VkBridge (#625) 2018-01-30 16:57:07 +00:00
Tameroski be03764029 Fixing double quote issue at the end of URL (#623) 2018-01-23 11:27:45 +00:00
Matt DeMoss a07874d468 Initial commit for Bloomberg bridge with top stories and search (#607)
* initial commit for Bloomberg bridge with top stories and search
2018-01-12 12:08:15 +00:00
Matt DeMoss 90d7ae8776 Fix twitter list filter test #613, fix and change getName() for lists. (#614) 2018-01-12 12:07:40 +00:00
Teromene 93e0562353
Merge pull request #610 from mdemoss/YouTubeTitle-#609
You tube title fix for #609
2018-01-11 12:09:38 +00:00
Teromene 4c5d547d9c
Merge pull request #608 from mdemoss/PcGamerBridge
Pc gamer bridge
2018-01-11 12:08:10 +00:00
Teromene 9a3a64010f
Merge pull request #620 from RSS-Bridge/teromene-patch-2
Update MixCloudBridge.php
2018-01-11 11:48:29 +00:00
Teromene e59a6f4c9e
Update MixCloudBridge.php
Fix whitespace at start of line
2018-01-11 11:44:51 +00:00
Teromene 1506e68587
Merge pull request #619 from RSS-Bridge/teromene-patch-1
Update .travis.yml
2018-01-11 11:44:37 +00:00
Teromene 671cba4f68
Update .travis.yml
Try to fix build failure
2018-01-11 11:41:25 +00:00
Teromene 374eb8f4bf
Merge pull request #617 from adamchainz/patch-1
README - sort lists alphabetically
2018-01-10 14:05:05 +00:00
Adam Johnson 60f7a2b3e4
README - sort lists alphabetically
This makes them easier to scan and check "does rss-bridge support service X I'm interested in?" :)
2018-01-10 11:45:55 +00:00
Teromene 7744172c63
Merge pull request #616 from lalannev/patch-1
Update LegifranceJOBridge.php
2018-01-09 17:19:28 +00:00