Commit graph

324 commits

Author SHA1 Message Date
ORelio
ae45a8cfee [contents] Fix open_basedir warning (#832)
References #818
2018-09-15 14:46:11 +02:00
Quentin Delmas
b397a42876 version: Bump to 2018-09-09 2018-09-09 21:00:10 +01:00
ORelio
de8cee6a1c Catching up | [Main] Debug mode, parse utils, MIME | [Bridges] Add/Improve 20 bridges (#802)
* Debug mode improvements

 - Improve debug warning message
 - Restore error reporting in debug mode
 - Fix 'notice' messages for unset fields

* Add parsing utility functions

html.php
 - extractFromDelimiters
 - stripWithDelimiters
 - stripRecursiveHTMLSection
 - markdownToHtml (partial)

bridges
 - remove now-duplicate functions
 - call functions from html.php instead

* [Anidex] New bridge

Anime torrent tracker

* [Anime-Ultime] Restore thumbnail

* [CNET] Recreate bridge

Full rewrite as the previous one was broken

* [Dilbert] Minor URI fix

Use new self::URI property

* [EstCeQuonMetEnProd] Fix content extraction

Bridge was broken

* [Facebook] Fix "SpSonsSoriSsés" label

... which was taking space in item title

* [Futura-Sciences] Use HTTPS, More cleanup

Use HTTPS as FS now offer HTTPS
Clean additional useless HTML elements

* [GBATemp] Multiple fixes

- Fix categories: missing "break" statements
- Restore thumbnail as enclosure
- Fix date extraction
- Fix user blog post extraction
- Use getSimpleHTMLDOMCached

* [JapanExpo] Fix bridge, HTTPS, thumbnails

- Fix getSimpleHTMLDOMCached call
- Upgrade to HTTPS as JE now offers HTTPS
- Restore thumbnails as enclosures

* [LeMondeInformatique] Fix bridge, HTTPS

- Upgrade to HTTPS as LMI now offers HTTPS
- Restore thumbnails using small images
- Fix content extraction
- Fix text encoding issue

* [Nextgov] Fix content extraction

- Restore thumbnail and use small image
- Field extraction fixes

* [NextInpact] Add categories and filtering by type

- Offer all RSS feeds
- Allow filtering by article type
- Implement extraction for brief articles
- Remove article limit, many brief articles are publied all at once

* [NyaaTorrents] New bridge

Anime torrent tracker

* [Releases3DS] Cache content, restore thumbnail

- Use getSimpleHTMLDOMCached
- Restore thumbnail as enclosure

* [TheHackerNews] Fix bridge

 - Fix content extraction including article body
 - Restore thumbnail as enclosure

* [WeLiveSecurity] HTTPS, Fix content extraction

- Upgrade to HTTPS as WLS now offers HTTPS
- Fix content extraction including article body

* [WordPress] Reduce timeout, more content selectors

- Reduce timeout to use default one (1h)
- Add new content selector (articleBody)
- Find thumbnail and set as enclosure
- Fix <script> cleanup

* [YGGTorrent] Increase limit, use cache

- Increase item limit as uploads are very frequent
- Use getSimpleHTMLDOMCached

* [ZDNet] Rewrite with FeedExpander

- Upgrade to HTTPS as ZD now offers HTTPS
- Use FeedExpander for secondary fields
- Fix content extraction for article body

* [Main] Handle MIME type for enclosures

Many feed readers will ignore enclosures (e.g. thumbnails) with no MIME type. This commit adds automatic MIME type detection based on file extension (which may be inaccurate but is the only way without fetching the content).

One can force enclosure type using #.ext anchor (hacky, needs improving)

* [FeedExpander] Improve field extraction

- Add support for passing enclosures
- Improve author and uri extraction
- Fix 'notice' PHP error messages

* [Pull] Coding style fixes for #802

* [Pull] Implementing changes for #802

 - Fix coding style issues with str append
 - Remove useless CACHE_TIMEOUT
 - Use count() instead of $limit
 - Use defaultLinkTo() + handle strings
 - Use http_build_query()
 - Fix missing </em>
 - Remove error_reporting(0)
 - warning CSS (@LogMANOriginal)
 - Fix typo in FeedExpander comment

* [Main] More documentation for markdownToHtml

See #802 for more details
2018-09-09 20:20:13 +01:00
Eugene Molotov
422c125d8e [core] Returning 304 http code when returning cached data (#793) 2018-08-25 20:00:38 +01:00
Quentin Delmas
be3620acb7 Add extension check for the "json" extension. 2018-08-22 16:21:20 +01:00
Walter Barrett
704a87ad97 Icons: Allow Bridge-specified icons (#788) 2018-08-21 17:46:47 +02:00
logmanoriginal
de7622ebbf version: Bump to 2018-08-07 2018-08-07 18:37:38 +02:00
Eugene Molotov
df58f5bbdb [core] Add urljoin (#756)
Adds php-urljoin from https://github.com/fluffy-critter/php-urljoin to replace the custom implementation of 'defaultLinkTo'
2018-08-02 06:31:56 +02:00
logmanoriginal
65ec04ea98 [contents] Remove superfluous debug log from getContents
References #757
2018-07-25 19:56:46 +02:00
LogMANOriginal
d83f2f285b
Separate index and bridge card generating code into a separate classes (#734)
[html] Generate index and bridge cards using separate clases

Move HTML generating code from 'index.php' to 'Index.php', separating components into static functions.

Move HTML generation code for bridge cards from 'html.php' to 'BridgeCard.php', separating components into static functions.
2018-07-21 18:15:07 +02:00
logmanoriginal
066ef1d7db [contents] Add Cloudflare challenge detection
Adds detection for servers responding with Cloudflare challenges,
throwing a server error if detected:

"The server responded with a Cloudflare challenge, which is not
supported by RSS-Bridge! If this error persists longer than a week,
please consider opening an issue on GitHub!"

This is supposed to support maintainers to identify broken bridges
for sites with Cloudflare enabled permanently. It doesn't circumvent
the protection in any form or shape!

The Cloudflare challenge is detected by analyzing the last response
header received from the server. If the HTTP Code is not 200 (OK)
and the server name contains 'cloudflare' ('Server: cloudflare'),
RSS-Bridge assumes the server responded with a challenge.

The header parsing is based on https://stackoverflow.com/a/18682872
2018-07-21 17:43:29 +02:00
logmanoriginal
6a98293fb3 [Configuration] Bump version to 2018-07-17 2018-07-17 20:44:01 +02:00
logmanoriginal
d79630e3b8 [Configuration] Remove check for allow_url_fopen
This commit follows the changes done in commits

fbf874cb29
ead7b2e8de
2018-07-17 20:39:14 +02:00
logmanoriginal
90dc968fd1 Fix PHPCS error 2018-06-30 10:26:48 +02:00
Teromene
da6b98851c Add recuperation of the current version from git if available (#731)
* Add recuperation of the current version from git if available
* Include version when auto-reporting an error
2018-06-30 10:24:22 +02:00
LogMANOriginal
193ca87afa [phpcs] enforce single quotes (#732)
* [phpcs] Add rule to enforce single quoted strings
2018-06-29 22:55:33 +01:00
Teromene
937ea49271 Add basic authentication support (#728)
* Move configuration in its own class in order to reduce the verbosity of index.php
* Add authentication mechanism using HTTP auth
* Add a method to get the config parameters
* Remove the installation checks from the index page
* Log all failed authentication attempts
2018-06-27 19:09:41 +02:00
hunhejj
d92da8f0f7 Add cUrl error message and code to the debugMessage (#711) 2018-06-10 22:08:45 +02:00
logmanoriginal
bc28c5da8e [contents] Set CURLOPT_HTTPHEADER only if the provided array contains data 2018-04-06 20:42:19 +02:00
logmanoriginal
5bd9c1611d [contents] Limit cURL protocols to HTTP and HTTPS 2018-04-06 20:42:19 +02:00
logmanoriginal
ee78e7613f [contents] Replace file_get_contents by cURL
cURL is a powerful library specifically designed to connect to many
different types of servers with different types of protocols. For
more detailed information refer to the PHP cURL manual:

- http://php.net/manual/en/book.curl.php

Due to this change some parameters for the getContents function were
necessary (also applies to getSimpleHTMLDOM and getSimpleHTMLDOMCached):

> $use_include_path removed

  This parameter has never been used and doesn't even make sense in
  this context; If set to true file_get_contents would also search
  for files in the include_path (specified in php.ini).

> $context replaced by $header and $opts

  The $context parameter allowed for customization of the request in
  order to change how file_get_contents would acquire the data (i.e.
  using POST instead of GET, sending custom header, etc...)

  cURL also provides facilities to specify custom headers and change
  how it communicates to severs. cURL, however, is much more advanced.

  - $header is an optional parameter (empty by default). It receives
    an array of strings to send in the HTTP request header.

    See 'CURLOPT_HTTPHEADER':

    "An array of HTTP header fields to set, in the format
    array('Content-type: text/plain', 'Content-length: 100')"

    - php.net/manual/en/function.curl-setopt.php

  - $opts is an optional parameter (empty by default). It receives
    an array of options, where each option is a key-value-pair of
    a cURL option (CURLOPT_*) and it's associated parameter. This
    parameter accepts any of the CURLOPT_* settings.

    Example (sending POST instead of GET):

    $opts = array(
      CURLOPT_POST => 1,
      CURLOPT_POSTFIELDS => '&action=none'
    );
    $html = getContents($url, array(), $opts);

    Refer to the cURL setopt manual for more information:
    - php.net/manual/en/function.curl-setopt.php

> $offset and $maxlen removed

  These options were supported by file_get_contents, but there doesn't
  seem to be an equivalent in cURL. Since no caller uses them they are
  safe to remove.

Compressed data / Encoding

  By using cURL instead of file_get_contents RSS-Bridge no longer has
  to handle compressed data manually.

  See 'CURLOPT_ENCODING':

  "[...] Supported encodings are "identity", "deflate", and "gzip".
  If an empty string, "", is set, a header containing all supported
  encoding types is sent."

  - http://php.net/manual/en/function.curl-setopt.php

  Notice: By default all encoding types are accepted (""). This can
  be changed by setting a custom option via $opts.

    Example:

    $opts = array(CURLOPT_ENCODING => 'gzip');
    $html = getContents($url, array(), $opts);

Proxy

The proxy implementation should still work, but there doesn't seem
to be an equivalent for 'request_fulluri = true'. To my understanding
this isn't an issue because cURL knows how to handle proxy communication.
2018-04-06 20:42:19 +02:00
teromene
ac6847045c Catch Errors in order to display a message in more cases. We also catch Exceptions to maintain compat with php 5.
Add check for simplexml extension.
2018-04-04 19:02:40 +01:00
LogMANOriginal
8ba817478b
Implement customizable cache timeout (#641)
* [BridgeAbstract] Implement customizable cache timeout

The customizable cache timeout is used instead of the default cache
timeout (CACHE_TIMEOUT) if specified by the caller.

* [index] Add new global parameter '_cache_timeout'

The _cache_timeout parameter is an optional parameter that can be
used to specify a custom cache timeout. This option is enabled by
default.

It can be disabled using the named constant 'CUSTOM_CACHE_TIMEOUT'
which supports two states:

> true: Enabled (default)
> false: Disabled

* [BridgeAbstract] Change scope of 'getCacheTimeout' to public

* [html] Add cache timeout parameter to all bridges

The timeout parameter only shows if CUSTOM_CACHE_TIMEOUT has been set
to true. The default value is automatically set to the value specified
in the bridge.

* [index] Disable custom cache timeout by default
2018-03-14 18:06:36 +01:00
logmanoriginal
1b1ab6a66e [validation] Fix error on undefined optional numeric value
Providing no value for an optional numeric parameter results in
error "Parameter *** is invalid!"

This is caused by the validation function ignoring the 'required'
attribute when loading and checking input parameters.

This commit adds checks to determine whether the 'required' attri-
bute is defined and active before returning the error message.

References #570:
2017-08-17 19:02:50 +02:00
logmanoriginal
4fb1366aaf [FeedExpander] Fix Serialization of 'SimpleXMLElement' is not allowed 2017-08-10 13:35:19 +02:00
logmanoriginal
8166e33e7f [FeedExpander] Remove whitespace from source content
Whitespace at the beginning of feeds causes parsing errors. This is
an example using an ill-formatted RSS feed:

   "XML or text declaration not at start of entity"
-- https://validator.w3.org

This commit automatically removes all proceeding and trailing white-
space from the source content before resume parsing.
2017-08-10 13:20:35 +02:00
logmanoriginal
4924769549 [validation] Remove superfluous if-statement 2017-08-06 13:45:24 +02:00
logmanoriginal
e4fa963bdf [validation] Return null on invalid number 2017-08-06 13:43:23 +02:00
logmanoriginal
99e7e7876e exception: Use built-in HTTP response codes
PHP >= 5.4 provides a built-in function to generate valid HTTP
error header including the error description: http_response_code()

See: http://php.net/manual/en/function.http-response-code.php
See also: https://stackoverflow.com/a/12018482

This commit removes the '\Http' utility class and replaces all
calls to 'Http::getMessageForCode()' by 'http_response_code()'
2017-08-06 12:55:11 +02:00
logmanoriginal
62c190d841 [Bridge] Remove superfuous variables and statements 2017-08-06 00:04:07 +02:00
logmanoriginal
84d2c02a09 whitelist: Do case-insensitive whitelist matching
Matching whitelisted bridges using a case-insensitive match makes
sense for following reasons:

- Wrong upper/lower case spelling in the whitelist is not easily
discovered. Example: Misspelling 'Youtube' as 'YouTube' will not
show the 'Youtube' bridge (while it is expected to show)

- Two bridges with the same name but different letter casing are
discouraged to prevent confusion and keep the project compatible
with Windows machines
2017-08-06 00:01:32 +02:00
logmanoriginal
a4b9611e66 [phpcs] Add missing rules
- Do not add spaces after opening or before closing parenthesis

  // Wrong
  if( !is_null($var) ) {
    ...
  }

  // Right
  if(!is_null($var)) {
    ...
  }

- Add space after closing parenthesis

  // Wrong
  if(true){
    ...
  }

  // Right
  if(true) {
    ...
  }

- Add body into new line
- Close body in new line

  // Wrong
  if(true) { ... }

  // Right
  if(true) {
    ...
  }

Notice: Spaces after keywords are not detected:

  // Wrong (not detected)
  // -> space after 'if' and missing space after 'else'
  if (true) {
    ...
  } else{
    ...
  }

  // Right
  if(true) {
    ...
  } else {
    ...
  }
2017-07-29 19:55:12 +02:00
LogMANOriginal
38b56bf23a [index] Improve error handling (#555)
Add additional information to error message:

- Name of the bridge
- Possible solutions
- Error description
- Error code
- Error message

* Output type changed from 'text' to 'html'
* Added styles for the error page
* Added a button to remotely open a GitHub issue

Closes #525
2017-07-29 19:16:16 +02:00
Frans de Jonge
781e4f1908 [FeedExpander] Deal with empty item 2017-06-24 15:09:15 +02:00
logmanoriginal
28331e7cd6 [BridgeAbstract] Return cached infos when using cached items
Re-requesting the same feed in normal mode (not debug mode)
returns the cached version. The name and URI of the feed
however was not returned.

This adds checks to getName() and getURI() in order to return
the cached infos when using the cached version.

Notice: For this to work correctly all bridges must call to
parent::getName() and parent::getURI() respectively if the
queriedContext is not defined. Example:

switch($this->queriedContext){
	case 'My context':
		// do your stuff here!
		break;
	default: parent::getName();
}
2017-04-23 21:06:25 +02:00
Teromene
59025d96bc Add an indicator to show the HTTP status of the site. (#483) 2017-03-18 19:02:18 +00:00
logmanoriginal
a2108c784f [FeedExpander] Properly cast simplexml elements
This fixes a possible cause of
"Serialization of 'SimpleXMLElement' is not allowed"
reported via #487
2017-03-13 22:12:11 +01:00
Teromene
8b2fdb3937 Add a function to convert the background-image attribute to an actual image. 2017-03-03 14:13:29 +00:00
logmanoriginal
16bdf6b204 links: Rename defaultImageSrcTo to defaultLinkTo
This function not only fixes image sources, but also anchors
2017-02-18 13:41:45 +01:00
logmanoriginal
cf7da1d41c [html] Fix anchors after fixing images
Anchors will be fixed in a similar way as it is done with images,
so it can be done in one go.
2017-02-18 13:40:58 +01:00
logmanoriginal
bb8e7495d8 [html] Fix img src replacement not working
strpos returns false if the needle was not found. See:
http://php.net/manual/en/function.strpos.php#refsect1-function.strpos-returnvalues
2017-02-18 13:13:40 +01:00
logmanoriginal
512a4f292b bridges: Return parent::getURI by default 2017-02-15 19:38:32 +01:00
logmanoriginal
c4169f1579 bridges: Return parent::getName by default 2017-02-15 19:38:32 +01:00
logmanoriginal
d93d491d8e core: Use methods to access bridge information
Bridge information were exposed and accessed via public constants
which doesn't work if you want to generate bridges dynamically as
discussed in #402
2017-02-15 19:38:32 +01:00
logmanoriginal
c44fb25845 core: Improve documentation and style for BridgeAbstract and BridgeInterface
Public functions defined in BridgeAbstract also belong to BridgeInterface

getInput may only be used by this class or its children.
2017-02-15 19:36:29 +01:00
logmanoriginal
ff83410534 style: Fix coding styles 2017-02-14 17:28:07 +01:00
Badet Aurélien
670d8f18cb [BridgeAbstract] Enable caching of extraInfos - Issue #431 (#434)
Enable caching of extraInfos.
2017-01-03 10:28:47 +00:00
logmanoriginal
4a1e5245b3 [contents] Don't suppress errors returned by file_get_contents 2016-12-15 20:24:34 +01:00
LogMANOriginal
cad78be37b Merge pull request #445 from da2x/patch-1
Set “Accept-Encoding: gzip” header
2016-12-14 12:57:48 +01:00
Daniel Aleksandersen
278d6a0ec2 Set “Accept-Encoding: gzip” header 2016-12-12 11:29:34 +01:00
Alexis CHEMEL
f4aa3b39e8 fix BridgeAbstract 2016-12-06 01:01:07 +01:00
Badet Aurélien
c702a0e69f Bridge getExtraInfos (#432)
* add function getExtraInfos() to BridgeAbstract

* replace call to $bridge->getName() and $bridge->getURI() by $bridge->getExtraInfos()

replace call to $bridge->getName() and $bridge->getURI() by $bridge->getExtraInfos() defined by default in BridgeAbstract.
So we could pass additionals ExtraInfos from custom bridges to custom formats.
2016-11-29 00:48:59 +00:00
logmanoriginal
14c689e7a3 [core] Fix typos 2016-11-09 19:10:40 +01:00
logmanoriginal
4f4fb11789 [FormatAbstract] Allow child classes to overwrite DEFAULT_CHARSET
By using 'static' instead of 'self' the constant may
be overridden by child classes.
2016-11-09 18:41:25 +01:00
logmanoriginal
28e813620f [FormatInterface] Add missing public functions
This commit adds all missing functions to the interface
that are defined and implemented as public functions in
FormatAbstract.
2016-11-09 18:41:24 +01:00
logmanoriginal
49281a2ed3 [FeedExpander] Remove orphan getDescription function 2016-10-16 12:47:37 +02:00
logmanoriginal
b6feda2377 [contents] Use FileCache for getSimpleHTMLDOMCached 2016-10-08 16:30:01 +02:00
logmanoriginal
5c309e93dc [cache] Specify cache duration for 'purgeCache' 2016-10-08 16:18:10 +02:00
logmanoriginal
5de4a59d41 [index] Initialize cache before loading to bridge
Previously BridgeAbstract needed to know which exact implementation
of CacheInterface was used (since we only got one right now its
not a problem). Initializing the cache in index.php instead allows
to change cache types more easily.
2016-10-08 15:29:21 +02:00
logmanoriginal
5ccde61a19 [FileCache] Rename 'prepare' to 'setParameters'
This is a cosmetic change to use the same naming convention for
all methods.
2016-10-08 15:04:14 +02:00
logmanoriginal
0998cbde9d [cache] Directly implement CacheInterface in FileCache
The function 'prepare' previously implemented in CacheAbstract
is specifically required for FileCache and thus belongs to FileCache.
Since this change removes all code from CacheAbstract, it can be
removed completely.
2016-10-08 14:52:03 +02:00
logmanoriginal
9ac678aac5 [Cache] Move 'purge' function to implementations
The purge function is cache specific and thus belongs
to the specific implementation.
2016-10-07 22:33:45 +02:00
logmanoriginal
51ff8de346 [Cache] Remove orphan function utf8_encode_deep 2016-10-07 22:06:58 +02:00
logmanoriginal
ee7ddcf992 [FeedExpander] Fix SimplXMLElement serialization error
Previously FeedExpander stored SimpleXMLElement objects into
the items array. These objects cannot be serialized using the
serialize function.
2016-10-02 18:07:56 +02:00
logmanoriginal
b0784e5a42 [BridgeAbstract] Don't load cache file in DEBUG mode 2016-10-02 17:00:53 +02:00
logmanoriginal
970e216c1f Merge branch 'methods2functions' of https://framagit.org/peetah/rss-bridge 2016-10-02 16:05:58 +02:00
Pierre Mazière
a1ef589ffe [core] extract BridgeAbstract methods to make them functions
Method validateData is now afunction in lib/validation.php
validateTextValue, validateNumberValue, validateCheckboxValue
and validateListValue are now anonymous functions
defined in validateData

Signed-off-by: Pierre Mazière <pierre.maziere@gmx.com>
2016-09-26 00:09:29 +02:00
Pierre Mazière
747be13849 [core] extract HTMLUtils methods to make them functions
Methods displayBridgeCard, sanitize, defaultImageSrcTo are now
functions in lib/html.php
getHelperButtinsFormat and getFormHeader are now anonymous functions
defined in displayBridgeCard

Signed-off-by: Pierre Mazière <pierre.maziere@gmx.com>
2016-09-25 23:58:52 +02:00
Pierre Mazière
f1fb95b257 [core] extract BridgeAbstract methods to make them functions
- returnError, returnServerError, returnClientError ,debugMessage are
  moved to lib/error.php

- getContents, getSimpleHTMLDOM, getSimpleHTMLDOMCached are moved to
  lib/contents.php

Signed-off-by: Pierre Mazière <pierre.maziere@gmx.com>
2016-09-25 23:22:33 +02:00
Pierre Mazière
235c985901 [BridgeAbstract] replace getCacheDuration method by CACHE_TIMEOUT class constant
Signed-off-by: Pierre Mazière <pierre.maziere@gmx.com>
2016-09-25 17:04:28 +02:00
logmanoriginal
351eb00400 [FeedExpander] Align logical operators in next line
This is a cosmetic change to apply the same standard as
in HTMLUtils
2016-09-17 20:44:31 +02:00
logmanoriginal
2331c4fbab Merge branch 'bridges2feeds' of https://framagit.org/peetah/rss-bridge 2016-09-17 20:34:12 +02:00
logmanoriginal
ac3a2b8e6e [BridgeAbstract] Fix typos 2016-09-17 20:24:59 +02:00
Pierre Mazière
70c490d6ef [BridgeAbstract] handle compressed data
Signed-off-by: Pierre Mazière <pierre.maziere@gmx.com>
2016-09-17 19:37:41 +02:00
Pierre Mazière
ca0842ccf8 [FeedExpander] widen guid use as uri provider
Signed-off-by: Pierre Mazière <pierre.maziere@gmx.com>
2016-09-17 19:37:39 +02:00
Pierre Mazière
2744c13735 [FeedExpander] fix feeds using guid tag as item uri provider
Signed-off-by: Pierre Mazière <pierre.maziere@gmx.com>
2016-09-17 19:37:39 +02:00
logmanoriginal
1819943451 [FeedExpander] Write debug message for custom build function 2016-09-17 18:19:26 +02:00
logmanoriginal
ffc9418620 [FeedExpander] Fix typos 2016-09-17 18:16:25 +02:00
logmanoriginal
a333226733 Merge branch 'autodetectFeedExpander' of https://framagit.org/peetah/rss-bridge 2016-09-17 18:09:05 +02:00
Pierre Mazière
15c422c648 [FeedExpander] implement default parseItem() method
Signed-off-by: Pierre Mazière <pierre.maziere@gmx.com>
2016-09-12 10:39:34 +02:00
Pierre Mazière
655b3d578d [FeedExpander] simplify feed type detection and store it
Signed-off-by: Pierre Mazière <pierre.maziere@gmx.com>
2016-09-12 10:38:50 +02:00
logmanoriginal
6943abd9c2 [Bridge] Fix whitelist check 2016-09-11 14:02:02 +02:00
logmanoriginal
ab16af631e [core] Apply some fixes 2016-09-10 21:01:02 +02:00
logmanoriginal
62eec43980 [core] Apply common indentation
All files are now using tabs for indentation
2016-09-10 20:41:11 +02:00
logmanoriginal
9be03f199b [core] Remove HttpCachingBridgeAbstract
BridgeAbstract implements all functions to cover the implementation
2016-09-10 19:13:01 +02:00
logmanoriginal
2eec89ab27 [bridges] Change all bridges to use BridgeAbstract with getSimpleHTMLDOMCached 2016-09-10 19:11:09 +02:00
logmanoriginal
9f2dd48684 [BridgeAbstract] Add getSimpleHTMLDOMCached
This function is a copy of the get_cached function from
HttpCachingBridgeAbstract, adding all parameters of
getSimpleHTMLDOM in order to replace the need of
HttpCachingBridgeAbstract entirely
2016-09-10 19:04:01 +02:00
logmanoriginal
33584b8423 [HttpCachingBridgeAbstract] General cleanup
- Remove buildCacheFilePath -> Single use, so no real purpose
- Simplify debug messages
- Cleanup documentation
2016-09-10 00:34:25 +02:00
logmanoriginal
3a92a1e04b [HttpCachingBridgeAbstract] Remove get_cached_time and remove_from_cache
Bridges no longer require to check cache file durations
manually as get_cached now supports the duration parameter
2016-09-10 00:09:10 +02:00
logmanoriginal
1a2a48e129 [HttpCachingBridgeAbstract] Use sha1 cache file names
Previously folder hierarchies were created based on the input URL
which could result in very deep folder structures. Using SHA1 file
names also allows to check for old files more effectively (via
fileatime)
2016-09-10 00:00:10 +02:00
logmanoriginal
ce00c6f869 [HttpCachingBridgeAbstract] Don't pass cache directory as parameter 2016-09-09 22:23:40 +02:00
logmanoriginal
84956c4daf [HttpCachingBridgeAbstract] Add duration parameter to 'get_cached'
This allows to specify the cache duration for a specific
url without the need to delete the cache file first.
Instead the cache file is automatically deleted if the
duration elapsed.
2016-09-09 22:23:15 +02:00
logmanoriginal
f1fb527607 [FeedExpander] Add optional parameter to specify max items
Allows caller of collectExpandableDatas to request a limited
amount of items
2016-09-05 20:17:00 +02:00
logmanoriginal
298dc49c67 [lib] Split Bridge/Cache/Format into one file per class
The files have grown to a size where it is necessary to search
for a class in a file. This commit splits the content into one
file per class. RSS-Bridge will require implementations and
the implementations will require (once) the interfaces.
2016-09-05 18:05:19 +02:00
logmanoriginal
acde8a2cea [Bridge] Support 'dc:date' for RSS 2.0 parser 2016-09-04 13:46:57 +02:00
logmanoriginal
546c0036df [Bridge] Rename class RssExpander to FeedExpander
This class no is not RSS only anymore
2016-09-04 13:28:12 +02:00
logmanoriginal
39788485ea [Bridge] Rename 'parseRSSItem' to 'parseItem' 2016-09-04 13:26:17 +02:00
logmanoriginal
149b64879e [Bridge] Support 'dc' namespace for RSS 1.0 2016-09-04 13:05:17 +02:00
logmanoriginal
8fa0b9660f [Bridge] Fix ATOM feed uri detection 2016-09-04 12:40:42 +02:00
logmanoriginal
bf20a2f687 [Bridge] Remove uneccesary debug message 2016-09-04 12:34:56 +02:00
logmanoriginal
51a3a75aac [Bridge] Add RSS 1.0 and 2.0 parser 2016-09-04 12:32:56 +02:00
logmanoriginal
f781965825 [Bridge] Add RSS 0.91 parser 2016-09-04 11:45:14 +02:00
logmanoriginal
0e5775012f [Bridge] Rename parameter 'name' to 'url' 2016-09-04 11:01:13 +02:00
logmanoriginal
2aa9b8f026 [Bridge] Extend RssExpander to load ATOM formats 2016-09-03 22:17:36 +02:00
logmanoriginal
a8883523f4 [Bridge] Pass 'queriedContext' as parameter to 'setInputs'
This makes clear that 'setInputs' depends on the 'queriedContext'
2016-09-02 20:05:18 +02:00
logmanoriginal
d1ff23c7ba [Bridge] Fix incorrect switch variable 2016-09-02 19:53:59 +02:00
logmanoriginal
04bddd0758 [Bridge] Apply default value while setting inputs 2016-09-02 19:52:47 +02:00
logmanoriginal
5f3d60276a [Bridge] Use original input values to guess context 2016-09-02 19:46:58 +02:00
logmanoriginal
4bc4e03d7b [Bridge] Change order of variable declaration 2016-09-02 19:03:51 +02:00
logmanoriginal
41d3aa0695 [Bridge] Change scope of member variable 'inputs' 2016-09-02 19:01:57 +02:00
logmanoriginal
48db1693a1 [Bridge] Use member variable instead of local variable 2016-09-02 18:59:55 +02:00
logmanoriginal
c2e411ba82 [Bridge] Add function to set inputs 2016-09-02 18:38:07 +02:00
logmanoriginal
37f269cf53 [Bridge] Remove unused variable 2016-09-02 18:12:37 +02:00
logmanoriginal
d520e82d9e [Bridge] Fix function names
is* implies Boolean return values, which is not true for
these functions
2016-09-02 18:11:02 +02:00
logmanoriginal
5ad1fa8242 Merge branch 'constantine' of https://framagit.org/peetah/rss-bridge 2016-09-02 17:36:51 +02:00
Pierre Mazière
556b8a2452 [core] transform some BridgeAbstract members to class constants
This concerns $uri, $name, $maintainer, $parameters and $description

Signed-off-by: Pierre Mazière <pierre.maziere@gmx.com>
2016-09-01 23:12:51 +02:00
Pierre Mazière
729cbadc77 [Bridge] fix input parameter value storage
Signed-off-by: Pierre Mazière <pierre.maziere@gmx.com>
2016-09-01 23:12:30 +02:00
logmanoriginal
1aa01a1f67 [Bridge] Fix invalid text value without pattern 2016-09-01 21:07:27 +02:00
logmanoriginal
47b6cb8937 [Bridge] Return name of the parameter that failed the check 2016-09-01 20:59:49 +02:00
logmanoriginal
7c36b51aa2 [Bridge] Move code into functions 2016-09-01 20:53:47 +02:00
logmanoriginal
cd9435b936 [Bridge] Simplify cache loading 2016-09-01 19:43:25 +02:00
logmanoriginal
8d050c233b [Format] Change scope of 'sanitizeHtml' to protected 2016-08-29 20:51:11 +02:00
logmanoriginal
f49fca516d [Format] Trim all items elements
This removes unnecessary whitespace in output data
2016-08-29 20:50:02 +02:00
logmanoriginal
cf146523be [formats] Rename variable 'data' to 'item'
This makes the intend of the variable more clear and is now
coherent with all Bridges
2016-08-29 19:47:21 +02:00
logmanoriginal
a84016bcb6 [core] Rename item getter/setter
getDatas -> getItems
setDatas -> setItems

Note: Bridge->setDatas actually sets data, where Bridge->getItems
only returns items (this is why Bridge->setDatas was not changed)
2016-08-29 19:42:58 +02:00
Pierre Mazière
b1a834801b [BridgeAbstract] fix empty parameter context
Signed-off-by: Pierre Mazière <pierre.maziere@gmx.com>
2016-08-29 13:27:30 +02:00
logmanoriginal
8ed3b02899 Merge branch 'core' of http://framagit.org/peetah/rss-bridge 2016-08-28 20:33:27 +02:00
logmanoriginal
e4b314f78a [Bridge] Enable cache file deletion 2016-08-28 19:50:33 +02:00
logmanoriginal
5f3d28f3a6 [Bridge] Return HTML DOM with get_cached 2016-08-28 19:37:32 +02:00
Pierre Mazière
fb05976fa6 [core] fix two-levels list default value support
Signed-off-by: Pierre Mazière <pierre.maziere@gmx.com>
2016-08-28 13:05:03 +02:00
Pierre Mazière
74d876b585 [core] add explicit support for list default value
and implicit support for text and number parameters

Signed-off-by: Pierre Mazière <pierre.maziere@gmx.com>
2016-08-28 13:05:03 +02:00
Pierre Mazière
93634ada3a [core] really fix bridges without parameters
Signed-off-by: Pierre Mazière <pierre.maziere@gmx.com>
2016-08-28 13:05:03 +02:00
Pierre Mazière
16dc7fcb6f [core] fix bridges without parameters
Signed-off-by: Pierre Mazière <pierre.maziere@gmx.com>
2016-08-28 13:05:03 +02:00
Pierre Mazière
1b3c8a8aeb [core + bridges] add BridgeAbstract::$inputs and BridgeAbstract::getInput()
Inputs are not stored in BridgeAbstract::$parameters anymore to separate
static data from dynamic data.
The getInput method allows for more readable code.

Also fix an "undefined index 'global'" notice

Probability of breaking bridges: high !

Signed-off-by: Pierre Mazière <pierre.maziere@gmx.com>
2016-08-28 13:05:03 +02:00
Pierre Mazière
2047955f06 [core] rename $param to $inputs
Signed-off-by: Pierre Mazière <pierre.maziere@gmx.com>
2016-08-28 13:05:03 +02:00
Pierre Mazière
d1715f6efc [core] fix missing curly brackets
Signed-off-by: Pierre Mazière <pierre.maziere@gmx.com>
2016-08-28 13:05:03 +02:00
Pierre Mazière
85625318b1 [core] variable renaming
Signed-off-by: Pierre Mazière <pierre.maziere@gmx.com>
2016-08-28 13:05:03 +02:00
Pierre Mazière
ea1d53a24a [core] fix global parameter values propagatino
Signed-off-by: Pierre Mazière <pierre.maziere@gmx.com>
2016-08-28 13:05:03 +02:00
Pierre Mazière
701aff420d [core] code simplification
Signed-off-by: Pierre Mazière <pierre.maziere@gmx.com>
2016-08-28 13:05:03 +02:00
Pierre Mazière
de1b39c8e5 [core + bridges] get rid of loadMetadata
if a bridge needs to modify some of the data that were initialized
there, ::__construct() should be used instead.

Signed-off-by: Pierre Mazière <pierre.maziere@gmx.com>
2016-08-28 13:05:03 +02:00
Pierre Mazière
7857325d9c [core] set checkboxes default value to false
Signed-off-by: Pierre Mazière <pierre.maziere@gmx.com>
2016-08-28 13:00:55 +02:00
Pierre Mazière
3f36ca2a9b [core] unexpected input generates a fail
because paranoïa is the way to security !
Should this be optional ?

Signed-off-by: Pierre Mazière <pierre.maziere@gmx.com>
2016-08-25 17:52:44 +02:00
Pierre Mazière
a67a219bf9 [core] remove '$param' argument from RssExpander::collectExpandableDatas
Signed-off-by: Pierre Mazière <pierre.maziere@gmx.com>
2016-08-25 17:11:49 +02:00
Pierre Mazière
117031bf0f [core] store parameters values in BridgeAbstract::parameters
This way, any BridgeAbstract method can now have access to these values,
no only collectData

Signed-off-by: Pierre Mazière <pierre.maziere@gmx.com>
2016-08-25 01:24:53 +02:00
Pierre Mazière
dea37c8e34 [core] use BridgeAbstract::parameters to sanitize inputs
This should result in a lot of simplifications in the bridges
since data validation is now done upstream.

Signed-off-by: Pierre Mazière <pierre.maziere@gmx.com>
2016-08-25 01:20:12 +02:00
logmanoriginal
1d0b8f20f3 [Bridge] Fix missing function scope 2016-08-24 20:50:32 +02:00
logmanoriginal
95404b8fc4 [Bridge] Move 'Bridge' class at top of the file 2016-08-24 20:48:12 +02:00
logmanoriginal
b9b2428f63 [Bridge] Add function to build chache file path 2016-08-24 20:44:21 +02:00
logmanoriginal
458d1fbfeb [Bridge] Rename to 2016-08-24 20:35:19 +02:00
logmanoriginal
56cb116ce6 [Bridge] Format unreadable lines longer than 80 chars 2016-08-24 20:31:39 +02:00