Commit graph

1737 commits

Author SHA1 Message Date
teromene
3ca59392c2 Fix for crashes when accessing FileCache in case it has been purged/not created yet. 2018-05-05 18:05:48 +01:00
teromene
9b34b68180 Do not use an external service in order to fetch the favicon. 2018-05-05 13:55:38 +01:00
teromene
79ebdc4b39 Warn the user when trying to fetch a non-public facebook page. 2018-05-05 13:49:49 +01:00
teromene
8770c87389 Added support for stories in InstagramBridge. Closes #665
Renamed parameters as stories are only available in user mode.
Use a regex instead of HTML parsing to extract the JSON, as it is way faster.
2018-05-05 13:00:59 +01:00
Eugene Molotov
c1e3352218 [VkBridge] Extended article link parsing (#685)
* [VkBridge] Extended article link parsing
2018-05-05 12:03:54 +02:00
Grégory T
00570ce1b4 [ETTVBridge] New bridge, first push (#680)
* [ETTVBridge] New bridge
2018-04-30 23:18:39 +02:00
teromene
df33dcff4e [YGGTorrentBridge] URL encode the first parts of the requests. 2018-04-26 22:57:18 +01:00
Nicolas Delsaux
e60b5ab193 Mise à jour du bridge pour WorldOfTanks (#527)
* Mise à jour de l'un de mes bridges fétiches
2018-04-22 12:58:07 +02:00
teromene
b0c7a62f74 [index] Bumped version to 2018-04-20 2018-04-20 17:15:25 +02:00
teromene
57b15a089e Added DiscogsBridge. Closes #615 2018-04-20 16:57:09 +02:00
teromene
4b7fbe4188 DansTonChatBridge: test before accessing plaintext 2018-04-19 21:00:18 +02:00
Teromene
2390fb58b3
Merge pull request #673 from GregThib/patch-1
DansTonChatBridge: Update to follow DTC website changes
2018-04-19 20:58:01 +02:00
Teromene
1e8d29f6ec
Merge pull request #672 from em92/patch-3
[YoutubeBridge] Removed duration in titles on search mode
2018-04-19 20:56:34 +02:00
Eugene Molotov
644d13686c [YoutubeBridge] Removed duration in titles on search mode 2018-04-19 09:03:29 +05:00
teromene
aa0ff1c9b1 Added YGGTorrentBridge. 2018-04-18 21:57:27 +02:00
teromene
539d9f1f06 Add SupInfoBridge, fixes #668 2018-04-18 12:39:45 +02:00
teromene
5ece801ce7 Fix h* display size in HtmlFormat, and fix images being wider than the page. 2018-04-18 12:29:22 +02:00
GregThib
4dcea6d9c9
Update to follow DTC website changes
Now, entry title is optionnal and may be found in h3 HTML element.
Entry content is mandatory and may be found in div[class="item-content"] HTML element.

Moreover, the title may contain simple quotes (here, encoded) so the bridge have to decode first to apply format library function. In case we don't do that, the format function double encode the quote and something like ' could appear.
2018-04-18 12:00:00 +02:00
teromene
d69e2521f1 Removed T411 bridge. Website was closed nearly one year ago. 2018-04-18 11:44:54 +02:00
teromene
7927d73719 Rewrote DemonoidBridge. Fixes #626. 2018-04-17 15:25:02 +02:00
teromene
0620f30ae0 Changed the API key used for SoundCloud bridge. Should fix #599 2018-04-17 14:24:00 +02:00
teromene
795494cfce Added enclosures to InstagramBridge. 2018-04-16 19:34:21 +02:00
teromene
ba8542156c Remove usage of function file_get_contents. 2018-04-16 19:27:20 +02:00
Eugene Molotov
55f112e034 [VkBridge] Rewrited bridge code (#667)
* [VkBridge] Convert special HTML entities to characters in pageName

* [VkBridge] Generate feed item title

* [VkBridge] Remove double backslashes in feed item link

* [VkBridge] Unpin post if pinned

* [VkBridge] Mark reposted messages

* [VkBridge] Correct external link parsing

* [VkBridge] Added article parsing

* [VkBridge] Added video parsing

* [VkBridge] Added photo parsing

* [VkBridge] Added album link parsing

* [VkBridge] Added one more external link selector

* [VkBridge] Using array of link selectors to remove

* [VkBridge] Added document parsing

* [VkBridge] Added sign parsing

* [VkBridge] Fixed incorrect sorting with pinned item

* [VkBridge] More methods to parse documents

* [VkBridge] Save fallback if page name element not found

* [VkBridge] Using post signed as feed item author

* [VkBridge] Fixed document link

* [VkBridge] Coding policy fixes
2018-04-16 10:55:31 +01:00
Mitsukarenai
208fff801d [FDroid] minor fixes for Travis CI 2018-04-15 13:21:48 +02:00
Mitsukarenai
3c9860de43 [FDroid] new bridge 2018-04-15 13:13:10 +02:00
Adam Tygart
a16ec196c5 [NotAlways] Add a bridge for the NotAlways family of sites (#537)
NotAlways right found it necessary to remove their RSS feeds recently. This is a *simple* bridge to grab the ones on the front page. It allows you to filter the articles based on their classification (right, working, romantic, related, learning, friendly, hopeless, unfiltered, or all).
2018-04-15 12:02:37 +01:00
teromene
887fc7b037 Fix GoComics, website completely changed. Fixes #663 2018-04-14 18:15:44 +01:00
teromene
1bd4a40f71 Added GNOME Builder configuration to gitignore. 2018-04-14 18:15:13 +01:00
teromene
494169f959 Added bridge for Pixiv.
This bridge is slow, as caching of images is required (REFERER header required to access the full size images)
2018-04-14 16:19:35 +01:00
logmanoriginal
178177e787 [index] Push version to 2018-04-06 2018-04-06 22:45:33 +02:00
logmanoriginal
1cb83ccea3 [IPBBridge] Use limit for the number of items
The limit was used to specify the number of pages to return from a given
topic which resulted in the number of returned items variing between one
and however many entries are listed on one page.

This commit changes the implementation for the limit to keep loading more
pages until the specified limit is reached. Excessive elements are removed
in order to return the exact amount of items specified by the limit.

This behavior is closer to how other bridges are implemented and makes it
more natural to use without being too confusing. Existing queries must be
updated to account for the new limit.

References #657
2018-04-06 22:25:49 +02:00
sysadminstory
c899399569 [DealabsBridge] Follow the website changes (#660) 2018-04-06 21:25:41 +02:00
LogMANOriginal
0f93370e92
Merge pull request #654 from LogMANOriginal/cURL
Use cURL instead of file_get_contents
2018-04-06 20:49:58 +02:00
logmanoriginal
45c3dcb636 [VkBridge] Simplify header specification 2018-04-06 20:42:19 +02:00
logmanoriginal
ecfc220b10 [KernelBugTrackerBridge] Fix too many parameters requesting HTML DOM 2018-04-06 20:42:19 +02:00
logmanoriginal
4b3efed7ec [YoutubeBridge] Fix too many parameters when using HTML mode 2018-04-06 20:42:19 +02:00
logmanoriginal
bc28c5da8e [contents] Set CURLOPT_HTTPHEADER only if the provided array contains data 2018-04-06 20:42:19 +02:00
logmanoriginal
5bd9c1611d [contents] Limit cURL protocols to HTTP and HTTPS 2018-04-06 20:42:19 +02:00
logmanoriginal
6caca4946b bridges: Fix bridges with custom headers and options
This commit fixes bridges which called getContents, getSimpleHTMLDOM
or getSimpleHTMLDOMCached with custom settings.
2018-04-06 20:42:19 +02:00
logmanoriginal
ee78e7613f [contents] Replace file_get_contents by cURL
cURL is a powerful library specifically designed to connect to many
different types of servers with different types of protocols. For
more detailed information refer to the PHP cURL manual:

- http://php.net/manual/en/book.curl.php

Due to this change some parameters for the getContents function were
necessary (also applies to getSimpleHTMLDOM and getSimpleHTMLDOMCached):

> $use_include_path removed

  This parameter has never been used and doesn't even make sense in
  this context; If set to true file_get_contents would also search
  for files in the include_path (specified in php.ini).

> $context replaced by $header and $opts

  The $context parameter allowed for customization of the request in
  order to change how file_get_contents would acquire the data (i.e.
  using POST instead of GET, sending custom header, etc...)

  cURL also provides facilities to specify custom headers and change
  how it communicates to severs. cURL, however, is much more advanced.

  - $header is an optional parameter (empty by default). It receives
    an array of strings to send in the HTTP request header.

    See 'CURLOPT_HTTPHEADER':

    "An array of HTTP header fields to set, in the format
    array('Content-type: text/plain', 'Content-length: 100')"

    - php.net/manual/en/function.curl-setopt.php

  - $opts is an optional parameter (empty by default). It receives
    an array of options, where each option is a key-value-pair of
    a cURL option (CURLOPT_*) and it's associated parameter. This
    parameter accepts any of the CURLOPT_* settings.

    Example (sending POST instead of GET):

    $opts = array(
      CURLOPT_POST => 1,
      CURLOPT_POSTFIELDS => '&action=none'
    );
    $html = getContents($url, array(), $opts);

    Refer to the cURL setopt manual for more information:
    - php.net/manual/en/function.curl-setopt.php

> $offset and $maxlen removed

  These options were supported by file_get_contents, but there doesn't
  seem to be an equivalent in cURL. Since no caller uses them they are
  safe to remove.

Compressed data / Encoding

  By using cURL instead of file_get_contents RSS-Bridge no longer has
  to handle compressed data manually.

  See 'CURLOPT_ENCODING':

  "[...] Supported encodings are "identity", "deflate", and "gzip".
  If an empty string, "", is set, a header containing all supported
  encoding types is sent."

  - http://php.net/manual/en/function.curl-setopt.php

  Notice: By default all encoding types are accepted (""). This can
  be changed by setting a custom option via $opts.

    Example:

    $opts = array(CURLOPT_ENCODING => 'gzip');
    $html = getContents($url, array(), $opts);

Proxy

The proxy implementation should still work, but there doesn't seem
to be an equivalent for 'request_fulluri = true'. To my understanding
this isn't an issue because cURL knows how to handle proxy communication.
2018-04-06 20:42:19 +02:00
logmanoriginal
2df2623430 [index] Add 'curl' extension check 2018-04-06 20:42:19 +02:00
logmanoriginal
de5f850cdb [index] Fix indentation using tabs 2018-04-06 20:34:44 +02:00
teromene
ac6847045c Catch Errors in order to display a message in more cases. We also catch Exceptions to maintain compat with php 5.
Add check for simplexml extension.
2018-04-04 19:02:40 +01:00
logmanoriginal
df6da837dc [FacebookBridge] Return error if username starts with slash
Requesting a username with a leading slash would cause error 500
because the requested URI would contain two slashes in a row.

For example username "/test" would result in:
https://facebook.com//test

References #628
2018-03-23 21:23:30 +01:00
Eugene Molotov
41b7984a4e [YoutubeBridge] Playlist mode: faster feed generating if item count is less or equal to 15 (#648)
* [YoutubeBridge] Playlist mode: faster feed generating if item count is less or equal to 15
2018-03-19 12:41:52 +00:00
teromene
38c7e0272e Add hashtag support to InstagramBridge.
Fixes  #629
2018-03-19 12:29:24 +00:00
teromene
29c690dbcd Fix InstagramBridge, thanks to @pintassilgo comments.
Fixes #646
2018-03-19 12:17:42 +00:00
LogMANOriginal
8ba817478b
Implement customizable cache timeout (#641)
* [BridgeAbstract] Implement customizable cache timeout

The customizable cache timeout is used instead of the default cache
timeout (CACHE_TIMEOUT) if specified by the caller.

* [index] Add new global parameter '_cache_timeout'

The _cache_timeout parameter is an optional parameter that can be
used to specify a custom cache timeout. This option is enabled by
default.

It can be disabled using the named constant 'CUSTOM_CACHE_TIMEOUT'
which supports two states:

> true: Enabled (default)
> false: Disabled

* [BridgeAbstract] Change scope of 'getCacheTimeout' to public

* [html] Add cache timeout parameter to all bridges

The timeout parameter only shows if CUSTOM_CACHE_TIMEOUT has been set
to true. The default value is automatically set to the value specified
in the bridge.

* [index] Disable custom cache timeout by default
2018-03-14 18:06:36 +01:00
Eugene Molotov
cacbe90102 [YoutubeBridge] Sort playlist items by publication date (#643) 2018-03-13 11:24:40 +00:00