rogs/cleanmedia
rogs/cleanmedia
1
0
Fork 0
Code Issues 1 Pull Requests 1 Actions Packages Projects Releases Wiki Activity

Add requirements

Browse Source
This commit is contained in:
Sebastian Spaeth 2023-05-17 12:17:16 +00:00
parent 4984a5d1a4
commit 13b7ffcc5e
1 changed files with 62 additions and 54 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

116
README.md
View File

@ -1,55 +1,63 @@
Cleanmedia
==========
A poor man's data retention policy for dendrite servers.
USAGE
=====
Check the command line options with --help. You mainly pass it the dendrite
configuration file as a means to find a) the media directory and b) the postgres
credentials for the dendrite data base.
You can also pass in the number of days you want to keep remote media.
How it works:
-------------
cleanmedia scours the database for all entries in the media repository
where user_id is an empty string (that is, the media was not uploaded
by a local user). It then deletes all entries, thumbnails and media
files that have been created `DAYS` time ago. (with DAYS being
configurable via command line and a default of 30 days)
This includes a number of remote media that we might want to keep
(e.g. avatar images of users on remote home servers).
But the main idea behind focusing on remote media is that a server
should be able to refetch remote media in case it is needed. It would
also make sense to delete local media, but that is more
complicated. (possible scenarios: local media older than Y days, rooms
that have been left by all users and are thus "unreachable", rooms that
have been upgraded but have users left in it, media that has not been "accessed" the last Y days,....)
But finding out these things and setting all these policies will be
way more difficult and in some cases we do not have the information
we'd need (e.g. if a media is part of an avatar image, or when media
has been accessed the last time).
In addition it performs some sanity checks and warns if inconsistencies occur:
1) Are there thumbnails in the db that do not have
corresponding media file entries (in the db)?
Todo
----
- Sanity checks: Are files on the file system that the db does not
know about?
LICENSE
=======
This code is released under the GNU GPL v3 or any later version.
There is no warranty for correctness or data that might be
Cleanmedia
==========
A poor man's data retention policy for dendrite servers.
USAGE
=====
Check the command line options with --help. You mainly pass it the dendrite
configuration file as a means to find a) the media directory and b) the postgres
credentials for the dendrite data base.
You can also pass in the number of days you want to keep remote media.
How it works:
-------------
cleanmedia scours the database for all entries in the media repository
where user_id is an empty string (that is, the media was not uploaded
by a local user). It then deletes all entries, thumbnails and media
files that have been created `DAYS` time ago. (with DAYS being
configurable via command line and a default of 30 days)
This includes a number of remote media that we might want to keep
(e.g. avatar images of users on remote home servers).
But the main idea behind focusing on remote media is that a server
should be able to refetch remote media in case it is needed. It would
also make sense to delete local media, but that is more
complicated. (possible scenarios: local media older than Y days, rooms
that have been left by all users and are thus "unreachable", rooms that
have been upgraded but have users left in it, media that has not been "accessed" the last Y days,....)
But finding out these things and setting all these policies will be
way more difficult and in some cases we do not have the information
we'd need (e.g. if a media is part of an avatar image, or when media
has been accessed the last time).
In addition it performs some sanity checks and warns if inconsistencies occur:
1) Are there thumbnails in the db that do not have
corresponding media file entries (in the db)?
Requirements
----
- Python >= 3.8
- psycopg2
- yaml
Todo
----
- Sanity checks: Are files on the file system that the db does not
know about?
LICENSE
=======
This code is released under the GNU GPL v3 or any later version.
There is no warranty for correctness or data that might be
accidentally deleted. Assume the worst and hope for the best!