Experimental: when a scheduled update runs, skip looking for and removing...
Experimental: when a scheduled update runs, skip looking for and removing other scheduled updates for the same feed, before scheduling the next update. My profiling seems to indicate that searching...
View ArticleAdded ruby-prof gem to the bundle. It is used for profiling.
More details: https://github.com/ruby-prof/ruby-prof
View ArticleMaximum update interval changed from 6 to 12 hours. Minimum changed from 10 t...
Less frequent updates, but still should be good enough. This will lighten the load on the background server a bit.
View ArticleResolved relative asset paths for bower components.
Should solve issue with hopscotch images being requested from the wrong path.
View ArticleFixed tests. Tests now use the configured max and min update intervals,...
Fixed tests. Tests now use the configured max and min update intervals, instead of hardcoded values. This means that if the max/min values in the config are changed, the tests will still pass.
View ArticlePerformance improvement when fixing update schedules.
The sidekiq objects for the different named queues are retrieved just once, instead of once for every feed in the db. This should reduce the time obtaining a connection, hopefully reduce...
View ArticlePerformance improvement when fixing schedules.
Querying Redis for each feed is very slow. What we do instead is to get all IDs passed to scheduled updates, all at once for each queue. We have then an array of IDs in a ruby variable for each named...
View ArticleRewrite recurrence of UpdateRedisCacheStatsWorker.
This change is in line with recommendations from the Sidetiq guys: https://github.com/tobiassvn/sidetiq/wiki/Known-Issues#ice_cube Instead of minutely(10) to run the job every 10 minutes, they suggest...
View ArticleUpdate interfal limits are again 6 hours - 15 minutes.
I think I've solved the most important performance problems in the background server. These lower interval limits should be no problem.
View ArticleMark feeds as permanently unavailable after 1 month, not 1 week.
I'm seeing too many feeds become unavailable for more than a week, only to be available again after some time. 1 month is maybe more reasonable.
View ArticleFixed tests.
Tests now use the configured maximum time that a feed can be failing before being marked as permanently unavailable, instead of a hardcoded value of 1 week. This way tests will pass even if this value...
View ArticleWhen fixing schedules, make sure that there is exactly one scheduled update...
When fixing schedules, make sure that there is exactly one scheduled update for each available feed. FixSchedulesWorker now not only adds missing scheduled updates. It also counts how many times are...
View ArticleExplicitly indicate that we want Rails to raise an exception if the protectio...
Explicitly indicate that we want Rails to raise an exception if the protection from forgery validations fail for a request.
View ArticleWhen deleting old entries, iterate using .each instead of .find_each
Unfortunately .find_each is not very flexible. Specifically it doesn't allow setting any other order than "id asc". In this case I want a more complex order, because ID fields are usually...
View ArticleWhen exporting OPML subscriptions, do not attempt to order feeds and folders ...
The .order clauses was being ignored by the find_each anyway, removing them does not change how it was already working. As a result the exported OPML will be less nice to read for human eyes, but it's...
View ArticleMake sure that a feed if valid utf-8 when fetching, before parsing.
To guarantee this: - after the http GET, if the response current encoding is not valid, look at the HTTP content-type header. It usually has a "charset=xxx" at the end that indicates the actual...
View ArticleWhen encoding fetched feeds as utf-8, replace incorrect characters with "?"...
When encoding fetched feeds as utf-8, replace incorrect characters with "?" instead of rising an error.
View ArticleWhen fetching a feed, assume response is in the charset reported in the HTTP...
When fetching a feed, assume response is in the charset reported in the HTTP content-type header, and convert to utf-8. This fixes an error in which sometimes a response with encoding different from...
View ArticleRefined regex that extracts charset from HTTP content-type header, to account...
Refined regex that extracts charset from HTTP content-type header, to account for things like single or double quotes, extra whitespace or extra semicolons. Also if no valid charset can be extracted...
View ArticleBetter control the case in which no charset is present in the HTTP headers....
Better control the case in which no charset is present in the HTTP headers. UTF-8 is used by default in this case.
View ArticleBetter control the case in which no entry states are found when removing a...
Better control the case in which no entry states are found when removing a feed subscription, so that no error is raised. This could happen because of workers parallellism when destroying a user.
View ArticleReplaced activerecord "first" queries with the equivalent "find" and "find_by...
Cleaner code is happier code.
View ArticleWhen destroying a feed subscription, check that the feed actually exists...
When destroying a feed subscription, check that the feed actually exists before checking to delete folders, entry states and the like. During a user destruction the feed may already have been destroyed...
View ArticleRefined handling of encodings different from utf-8:
- when fetching a feed, an attempt is made to detect the encoding from the content-type HTTP header. If it's not possible, or the reported encoding is not valid, the response XML is parsed to attempt...
View ArticleChange the way in which a feed response is checked to see if it's blank.
It seems the rails .blank? method has issues with some feeds related to charsets, perhaps when there is a character invalid for the reported charset. The new way is more verbose but has no such issues.
View ArticleControl the case in which the encoding cannot be determined from the feed XML...
In this case utf-8 will be used by default.
View ArticleControl the case in which user is already subscribed to feed, when...
Control the case in which user is already subscribed to feed, when autodiscovery determines that feed being fetched is a duplicate. When fetching a feed, it autodiscovery determines that the feed being...
View ArticleWhen checking if the received feed response is HTML rather than a feed XML,...
When checking if the received feed response is HTML rather than a feed XML, use nil? instead of present? The rails present? method can cause trouble if there is an encoding issue, rising an error....
View ArticleShow a small caret to the right of the entry currently being hovered.
This is a WIP to implement keyboard shortcuts. The caret will visually indicate the user which entry currently has the focus.
View Article