Post by: Dhammañāṇa on March 04, 2019, 11:25:09 AM
accesstoinsight.eu 13297.37 MB 13300 MB 13301 MB
Aside of the still open question who to perform reasonable maintenance for longer times:
My person thought it would be the best to invest some time and effort into cleaning up ati.eu manually.
Backup of ati.eu (about a week old) exists and the implementation files are still local saved. Since there are no spam edits to be expected, it seems to be well if deleting all the "work in progress files" in the history directory.
The most history files are lesser edits on content but in regard of coding and since it is still time of implementation. My person thinks it's fine to simply delete as much as possible of the history files, starting with:
media-attic, all folders
attic, cs-rm, -km, -th, -ru
to be continued...
Post by: Dhammañāṇa on March 04, 2019, 11:59:13 AM
accesstoinsight.eu 13218.59 MB 13300 MB 13301 MB
No special performance errors detected after the action.
My person thinks it's good to simply progress in this way, taking on the huge folders en and de, which might take a longer time to be deleted manually.
Starting with de first.
Post by: Dhammañāṇa on March 04, 2019, 06:51:19 PM
accesstoinsight.eu 12445.88 MB 13300 MB 13301 MB
My persons guess is that about 9GB are used only to store implementation-edits history.
Post by: Moritz on March 04, 2019, 07:19:53 PM
Aramika 
Dieses neue Thema (bzw. diese/r Beitrag/e) wurde aus abgetrennten Beiträgen, ursprünglich in Cleaning up "our" monastery, sangham.net (http://forum.sangham.net/index.php?topic=9165.0), hinzugefügt. Für ev. ergänzende Informationen zur sehen Sie bitte das Ursprugsthema ein. Anumodana!
The new topic (or post/s) here are originaly from Cleaning up "our" monastery, sangham.net (http://forum.sangham.net/index.php?topic=9165.0). For eventual additionally information: please visit also the Topic of origin. Anumodana!
[Original post:]
Vandami Bhante _/\_
From the last mirror of ATI that I have downloaded on my pc it looks like the largest problem is the cache directory, using more than 6 GB. The attic seems already trimmed down a lot (1 GB, latest state downloaded, some days ago), and the total amount of pages without attic is also about 1 GB.
I will try to implement cache cleanup also into the cleanup script.
_/\_
Post by: Moritz on March 04, 2019, 07:26:28 PM
- data/pages: 1.1 GB
- data/media: 3.2 GB
- data/meta: 281 MB
- data/media_meta: 568 KB (almost nothing)
- data/attic: 2.2 GB
- data/media_attic: 2.2 MB (almost nothing)
- data/index: 17MB (was not built successfully, should be almost 100 MB when complete)
- data/cache: 6.4 GB (half of the total space on the server)
_/\_
Post by: Dhammañāṇa on March 04, 2019, 07:36:18 PM
Vandami Bhante _/\_
From the last mirror of ATI that I have downloaded on my pc it looks like the largest problem is the cache directory, using more than 6 GB. The attic seems already trimmed down a lot (1 GB, latest state downloaded, some days ago), and the total amount of pages without attic is also about 1 GB.
I will try to implement cache cleanup also into the cleanup script.
_/\_
Sadhu.
May Master Moritz, how ever, always look at own needed time first. What ever inspired instead of normal "useless" spend times, please not duties or needed livelihood times.
If not otherwise told by Nyom, Atma would just continue the "manual" deleting in Attic.
Post by: Dhammañāṇa on March 04, 2019, 07:43:06 PM
It takes a long time to get the total directory size from FTP. But from the last copy I made one or two days ago of the complete site via FTP, what I have on my disk now there is:
- data/pages: 1.1 GB
- data/media: 3.2 GB
- data/meta: 281 MB
- data/media_meta: 568 KB (almost nothing)
- data/attic: 2.2 GB
- data/media_attic: 2.2 MB (almost nothing)
- data/index: 17MB (was not built successfully, should be almost 100 MB when complete)
- data/cache: 6.4 GB (half of the total space on the server)
_/\_
My person thought so and great that sure now. Sadhu for care.
Post by: Moritz on March 04, 2019, 07:56:30 PM
Vandami Bhante _/\_
From the last mirror of ATI that I have downloaded on my pc it looks like the largest problem is the cache directory, using more than 6 GB. The attic seems already trimmed down a lot (1 GB, latest state downloaded, some days ago), and the total amount of pages without attic is also about 1 GB.
I will try to implement cache cleanup also into the cleanup script.
_/\_
Sadhu.
May Master Moritz, how ever, always look at own needed time first. What ever inspired instead of normal "useless" spend times, please not duties or needed livelihood times.
If not otherwise told by Nyom, Atma would just continue the "manual" deleting in Attic.
Thanks for care. But was not much time needed. Cache is cleared for now (everything older than 30 days), only using 8.9 MB anymore. (I still don't understand how Cache really works, but it seems also to be safe in general to just delete it completely.)
I will add this to the general cleanup script with an explanation later.
_/\_
Post by: Dhammañāṇa on March 04, 2019, 08:09:14 PM
- https://www.dokuwiki.org/plugin:cleanup
- https://www.dokuwiki.org/plugin:clearhistory
Post by: Moritz on March 04, 2019, 08:12:07 PM
Post by: Dhammañāṇa on March 04, 2019, 08:37:25 PM
accesstoinsight.eu 6008.75 MB 13300 MB 13301 MB
Attic de delete is finished.
Atma would progress with the other directories manual, now en
Post by: Moritz on March 04, 2019, 09:37:46 PM
One would just have to call this URL (here with some example parameters):
http://accesstoinsight.eu/cleanup.php?dir=en&keep=0&expire=0&pass=password
The parameters are:
- expire: keep files which are younger than this number of days. (Default is 30 if not specified.)
- keep: keep as many revisions from beginning and end (oldest and youngest) as specified here. (Default is 3.)
- dir: e.g. 'de' to only delete from 'attic/de'; leave empty to cleanup all attic directories. (Default is empty, i.e. clean the whole attic.)
- pass: password (I will send this by email)
This same script is also currently run automatically every eight hours with the default parameters.
_/\_
Post by: Dhammañāṇa on March 05, 2019, 06:34:01 AM
Post by: Dhammañāṇa on March 05, 2019, 11:16:15 AM
Nevertheless, the grow of cache seems to be fast: now up again to 7751.8 MB. 1,5GB in some hours with no real edits.
Maybe the use of ~NOCACHE~ on all pages, or those fine with it, might be good.
https://www.dokuwiki.org/caching
https://www.dokuwiki.org/devel:caching
Post by: Moritz on March 06, 2019, 03:14:52 AM
Nevertheless, the grow of cache seems to be fast: now up again to 7751.8 MB. 1,5GB in some hours with no real edits.It has grown fast in the first few hours of re-indexing, but from what I had seen has stopped growing at 4.5 GB. Not sure where 7751.8 MB comes from, but I think the value is wrong. (Getting directory size from FTP takes a long time for a large directory tree, because every sub-directory has to be read with a single FTP command and then all counted together. I think many FTP programs use some cache to remember the size of directories and some numbers might become outdated if not reading everything again.)
Maybe the use of ~NOCACHE~ on all pages, or those fine with it, might be good.
But in any case, the cache is big. Maybe worth trying to turn off cache.
Probably easiest to try is to just disable caching in the admin settings (http://www.accesstoinsight.eu/index?do=admin&page=config), by setting "maximum age for cache" to -1 (not to 0 (https://www.dokuwiki.org/config:cachetime)), so one can see how well everything works without cache, whether page loading is much slower etc., without changing aynthing in the files at first.
But maybe good to wait until indexing is finished to have a good estimation of the performance.
_/\_
Post by: Moritz on March 06, 2019, 09:27:11 AM
Maybe worth trying to turn off cache.
Probably easiest to try is to just disable caching in the admin settings (http://www.accesstoinsight.eu/index?do=admin&page=config), by setting "maximum age for cache" to -1 (not to 0 (https://www.dokuwiki.org/config:cachetime)), so one can see how well everything works without cache, whether page loading is much slower etc., without changing aynthing in the files at first.
But maybe good to wait until indexing is finished to have a good estimation of the performance.
_/\_
Now that indexing is finished, I have set "maximum age for cache" to -1 (was 60*60*24 before) and cleared the cache (was about 4.6GB), to test if one can see a big difference in speed.
If page loading seems too slow, one can set "maximum age for cache" back to 60*60*24 or anything else, so the cache will be built up again.
_/\_
Post by: Dhammañāṇa on March 06, 2019, 10:16:50 AM