Commit graph

2141 commits

Author SHA1 Message Date
LogMANOriginal 3d301fc4ee
[contents] Skip caching if the remote server requests no caching (#945)
* Skip caching if Cache-Control defines no-cache
* Skip caching if Cache-Control defines no-store
2018-11-28 17:36:28 +01:00
triatic 263e8872ea core: Don't use server variables in CLI mode (#939) 2018-11-26 18:33:51 +01:00
logmanoriginal 6e9c188a72 [GlassdoorBridge] Fix bridge is marked as executable
References #938
2018-11-26 18:31:25 +01:00
Roliga 49da67cb33 core: Automatically select a bridge based on a URL (#928)
* core: Add bridge parameter auto detection

This adds a new 'detect' action which accepts a URL from which an
appropriate bridge is selected and relevant parameters are extracted.
The user is then automatically redirected to the selected bridge.

For example to get a feed from: https://twitter.com/search?q=%23rss-bridge
we could send a request to:
'/?action=detect&format=Atom&url=twitter.com/search%3Fq%3D%2523rss-bridge'
which would redirect to:
'/?action=display&q=%23rss-bridge&bridge=Twitter&format=Atom'.

This auto detection happens on a per-bridge basis, so a new function
'detectParameters' is added to BridgeInterface which bridges may implement.
It takes a URL for an argument and returns a list of parameters that were
extracted, or null if the URL isn't relevant for the bridge.

* [TwitterBridge] Add parameter auto detection

* [BridgeAbstract] Add generic parameter detection

This adds generic "paramater detection" for bridges that don't have any
parameters defined. If the queried URL matches the URI defined in the
bridge (ignoring https://, www. and trailing /) an emtpy list of parameters is
returned.
2018-11-26 18:05:40 +01:00
sysadminstory b4dbd191d0 [ZoneTelechargementBridge] Switch to the new Website (#934)
* [ZoneTelechargementBridge] Switch to the new Website

The website zone-telechargement1.org decided that he will be using a new
domain at the end of november :
https://www.annuaire-telechargement.com/

The bridge uses the new domain but still uses the same filename and
class name to keep the existing feed working.
2018-11-20 16:23:17 +01:00
logmanoriginal e09f452426 [.gitattributes] Exclude files from git archive
Files with the option "export-ignore" are excluded from "git archive"
commands. Release files from GitHub will also ignore those files, so
packages are smaller and don't include unneccessary files.
2018-11-19 18:11:09 +01:00
LogMANOriginal 7b261d1cc2
[contents] Add server side caching for all requests (If-Modified-Since) (#889)
This commit adds a cache for 'getContents' to '/cache/server'. All
contents are cached by default (even in debug mode). If debug mode
is enabled, the cached data is overwritten on each request.

In normal mode RSS-Bridge adds the 'If-Modified-Since' header with
the timestamp from the previously cached data (if available) to the
request.

Find more information on 'If-Modified-Since' here:
https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/If-Modified-Since

If the server responds with "304 Not Modified", the cached data is
returned.

If the server responds with "200 OK", the received data is written
to the cache (creates a new cache file if it doesn't exist yet).

No changes were made for all other response codes.

Servers that don't support the 'If-Modified-Since' header, will
respond with "200 OK".

For servers that respond with "304 Not Modified", the required band-
width will decrease and RSS-Bridge will responding faster.

Files in the cache are forcefully removed after 24 hours.

Notice: Only few servers actually do support 'If-Modified-Since'.
Thus, most bridges won't be affected by this change.
2018-11-19 17:53:08 +01:00
logmanoriginal 96a518c9e7 [html] Remove todo as it is already implemented 2018-11-18 17:52:45 +01:00
logmanoriginal 0d2ea9a677 [html] Rename parameters for sanitize() 2018-11-18 17:43:34 +01:00
logmanoriginal 66e82e46db [html] Remove todo tags
It is not feasible to use a single 'substr' in the functions
2018-11-18 17:36:00 +01:00
logmanoriginal 54800fcc8d [html] Clarify meaning of strange find() parameter
simple_html_dom currently doesnt support "->find('*')", which is a
known issue: https://sourceforge.net/p/simplehtmldom/bugs/157/

The solution implemented by RSS-Bridge is to find all nodes WITHOUT
a specific attribute. If the attribute is very unlikely to appear
in the DOM, this is essentially returning all nodes.

This is the meaning behind

"->find('*[!b38fd2b1fe7f4747d6b1c1254ccd055e]')"
2018-11-18 17:32:07 +01:00
logmanoriginal 67004556e6 [BridgeCard] Use self:: instead of BridgeCard:: 2018-11-18 16:59:13 +01:00
logmanoriginal c6a7b9ac64 exception: Remove HttpException class
This class served no particular purpose (other than adding a
layer on top of Exception).
2018-11-18 16:53:21 +01:00
logmanoriginal dbffbd4d4e [FormatAbstract] Check content type before sending header 2018-11-18 16:30:34 +01:00
logmanoriginal 1c17ffb5c4 [FeedExpander] Add constants for feed types 2018-11-18 16:18:40 +01:00
logmanoriginal 326cfb21cf [FeedExpander] Rename $name to $title 2018-11-18 16:11:38 +01:00
logmanoriginal 8ab1fb86a9 [FeedExpander] Let collectExpandableDatas() return self 2018-11-18 16:03:32 +01:00
logmanoriginal a9ec3d0d1f [Configuration] Change scope of $config to private 2018-11-18 15:58:34 +01:00
logmanoriginal ac5bcb62ec [Configuration] Add documentation for defined constants 2018-11-18 15:52:28 +01:00
logmanoriginal f24ab8b51b [Configuration] Rename $category to $section in getConfig() 2018-11-18 15:45:17 +01:00
logmanoriginal 4348119adf [Configuration] Make file paths explicit 2018-11-18 15:41:43 +01:00
logmanoriginal fd4124cda2 [Configuration] Make class final
This class is essential to the core library of RSS-Bridge and must
not be extended. This improves predictability for the behaviour of
this class.
2018-11-18 15:34:16 +01:00
logmanoriginal 91f7405297 [Configuration] Throw exception creating objects of this class
This class only provides static functions.
2018-11-18 15:29:50 +01:00
logmanoriginal 85685b7758 [Authentication] Throw exception creating objects from this class
Callers must use Authentication::showPromptIfNeeded()
2018-11-18 15:20:43 +01:00
logmanoriginal 41d02554f3 [YGGTorrentBridge] Add URI to feed items
References #931
2018-11-18 09:41:14 +01:00
logmanoriginal c4550be812 lib: Add API documentation 2018-11-18 09:41:14 +01:00
Thibault Couraud b29ba5b973 [CrewbayBridge] Update bridge according to new crewbay.com website (#930) 2018-11-18 09:16:24 +01:00
logmanoriginal 254fe9212a [Debug] Fix debug mode reports indexing error
Error log reports "PHP Notice:  Undefined offset: 2 in /rss-bridge/
lib/Debug.php on line 112" if the array returned by debug_backtrace
does not contain 3 items.

This commit fixes the issue by always using the last element in the
backtrace "end($backtrace)".
2018-11-16 20:19:52 +01:00
triatic 3806895059 [FacebookBridge] Improve titles (#924)
A slightly improved version of #454 and #468 . Build titles from content rather than author + pre-content (which doesn't reflect anything useful).
2018-11-16 15:33:54 +01:00
triatic 599d438a0d [FacebookBridge] Decode all elements in $item (#925) 2018-11-16 15:25:58 +01:00
triatic e5a6baab96 [TwitterBridge] Decode HTML entities (#926)
Removes duplicate encoding like " (should be ").
2018-11-15 22:00:01 +01:00
logmanoriginal b47a30ecc1 [rssbridge] Improve documentation 2018-11-15 20:52:17 +01:00
logmanoriginal 860b36c1e3 [Debug] Use self:: instead of Debug:: inside the class 2018-11-15 20:28:26 +01:00
logmanoriginal 3d475572c6 [Debug] Improve documentation 2018-11-15 20:27:32 +01:00
logmanoriginal 59f2d755fe format: Refactor searchInformation
- Rename function to getFormatName
- Add documentation
- Rename variables
- Remove unused variables
2018-11-15 20:16:21 +01:00
logmanoriginal d7c374bd8c [Format] Add function isFormatName
Returns true if the provided format name is valid
2018-11-15 20:14:43 +01:00
logmanoriginal 6b6ab6486a [Format] Store real path to working directory 2018-11-15 20:06:45 +01:00
logmanoriginal 6c4e239f64 format: Refactor class Format 2018-11-15 20:06:23 +01:00
logmanoriginal 88b0656954 bridge: Rename listBridge to getBridgeNames 2018-11-15 19:43:23 +01:00
logmanoriginal 66b11b8c41 [Bridge] Fix typo 2018-11-15 19:38:14 +01:00
logmanoriginal 1b34d9860e [Cache] Check if class is instantiable 2018-11-15 19:36:01 +01:00
logmanoriginal 6e70d461e1 [Bridge] Add function isBridgeName
This function returns true if the provided name is a valid
bridge name.
2018-11-15 19:33:56 +01:00
logmanoriginal 0a92b5d29b [Bridge] Refactor Bridge::create to improve readability 2018-11-15 19:31:31 +01:00
logmanoriginal e3849f45ab [Bridge] Use slashes to enclose regex 2018-11-15 19:30:33 +01:00
logmanoriginal 3d9c4a3718 [Bridge] Improve working directory handling
- Initialize with null to prevent leaking configurations
- Check if the working directory is a directory
- Store the real path instead of raw data
- Add final path separator as expected by Bridge::create
2018-11-15 19:28:56 +01:00
logmanoriginal 5f146a257e [Bridge] Change visibility from private to protected 2018-11-15 19:24:43 +01:00
logmanoriginal 936688e08c [Bridge] Fix typos 2018-11-15 19:22:32 +01:00
logmanoriginal 4b5372638c [Bridge] Use self:: instead of Bridge:: inside the class 2018-11-15 19:19:04 +01:00
logmanoriginal 6f4a8f4d03 [Bridge] Rename to in setWorkingDir 2018-11-15 19:17:18 +01:00
logmanoriginal 39652bb050 [Bridge] Rename to 2018-11-15 19:16:37 +01:00