'Log File Viewer' keeping too many logs - how to configure?
- Anmelden oder Registrieren um Kommentare zu schreiben
Hi all,
I'm asking for someone else whose computer has slowed down with hundreds of files in 'Log File Viewer'.
I'm not sure how to help clear up the excessive number of logs, nor on how to configure it, so it doesn't save everything. My own 'Log File Viewer' doesn't have this problem, but I can't explain why the difference.
Hope my question makes sense, I am a bit at loss even on how to ask for help with this.
As usual, many thanks for any help.
There is guide to log files https://forums.debian.net/viewtopic.php?t=160774
Thank you for the link, grosbidepoilu!
I was hoping for a user friendly gui alternative, so I could also help non-tech users to help themselves.
I'm looking into it -- would the option of archiving, as well as restricting to 1GB, be a good way to go? I feel responsible, but am a non-tech user either.
Debian Administrator said "I restrict journal logs to 200M as I've never found the need for journal logs past the previous boot."
SystemMaxUse=200M
https://forums.debian.net/viewtopic.php?p=809670#p809670
Yes, I saw that and made me think.
Just in case, 1GB could be very helpful, as sometimes the previous couple of days can become important.
Very interesting to learn about it, thank you again for adding that link!
Reading what is repeated in the log is a good idea. It may be be a problem worth solving (not only to have the logs take less space).
Yes, I agree Magic Banana! Any suggestion to simplify the process (as in 1,2,3 :) would be greatly appreciated
Just skim the (recent) log and see if some lines are repeated over and over. It is probably the case. If you do not understand them, you can copy them in reply to this post.
Oh, I see what you mean. I understand, but that is not the problem.
The problem is the amount of days, weeks, even months of logs that have added up to a point that it takes a looooooong while for Log File Viewer to load.
It's very unusual, at least in my experience. In my computer, Log File Viewer opens instantly, as it has few days of files being loaded.
OK. To only keep the last GB of log, execute in a terminal:
$ sudo journalctl --vacuum-size=1G
Instead of a size, you can specify a time with --vacuum-time.
Thank you Magic Banana!! This is so helpful!
$ sudo journalctl --vacuum-size=1G
How do we specify time with --vacuum-time? If we want to keep one week of logs, for example.
Will the excess of extra months of saved files be removed automatically?
Or should we delete them manually? If so, how to do it safely? (It is not my computer and I want to make sure I don't mess up)
How do we specify time with --vacuum-time? If we want to keep one week of logs, for example.
That would be:
$ sudo journalctl --vacuum-time 1week
Will the excess of extra months of saved files be removed automatically?
Yes. The command even reports what files in /var/log/journal/* are deleted and the disk space freed.
Those commands are for immediate removal. If you want the journal to always stay under a given size, you need to define SystemMaxUse in /etc/systemd/journald.conf, as grosbidepoilu told you. Administrator privileges are necessary to modify that file (here with pluma; choose you favorite text editor):
$ SUDO_EDITOR=pluma sudoedit /etc/systemd/journald.conf
Briliant! Thank so much Magic Banana!!!
Magic Banana, I followed all to the letter, but the excess logs were not removed. The Log File Viewer is still taking ages to open, slowly loading the files (going from 3 Sep to today, 3 Nov)
I even executed both commands:
First:
$ sudo journalctl --vacuum-time 1week
Restarted, but saw no change, so executed the other option:
$ sudo journalctl --vacuum-size=1G
Again, no change, all logs are still there!
I am thinking of using BleachBit to delete the excess logs, but want to confirm that this would cause no harm... can you enlighten me please?
You may want to look at the output of: ls -l /var/log --time=birth | grep 'syslog'
It would tell you when the active log files were created. The journalctl manual page, which I borrowed at the library for a bit of light reading, says:
The symptoms described here may have appeared because log files are not rotated properly, which would mean that all log history is kept in a single active file instead of being archived and eventually deleted (or available for manual deletion), which could explain why the above options have no effect. And incidentally why the logs are so slow to load in mate-system-log (MATE Log File Viewer).
Thank you Prospero!
I run the command and got this:
$ ls -l /var/log --time=birth | grep 'syslog'
ls: invalid argument ‘birth’ for ‘--time’
Valid arguments are:
- ‘atime’, ‘access’, ‘use’
- ‘ctime’, ‘status’
Try 'ls --help' for more information.
So, with info from "--help", I tried this:
$ ls -l /var/log --full-time | grep 'syslog'
-rw-r----- 1 syslog adm 1097527 2024-11-03 19:30:01.122317630 +0000 auth.log
-rw-r----- 1 syslog adm 684513 2024-07-17 08:14:43.680848207 +0100 auth.log.1
-rw-r----- 1 syslog adm 28663 2024-05-19 10:41:44.000000000 +0100 auth.log.2.gz
-rw-r----- 1 syslog adm 36018 2024-04-11 08:41:07.000000000 +0100 auth.log.3.gz
-rw-r----- 1 syslog adm 5583 2024-02-15 10:17:30.000000000 +0000 auth.log.4.gz
-rw-r----- 1 syslog adm 15963794 2024-11-03 19:25:46.464932123 +0000 kern.log
-rw-r----- 1 syslog adm 9895000 2024-07-17 08:14:43.244848196 +0100 kern.log.1
-rw-r----- 1 syslog adm 1252836 2024-05-19 10:41:43.000000000 +0100 kern.log.2.gz
-rw-r----- 1 syslog adm 1803742 2024-04-11 08:41:08.000000000 +0100 kern.log.3.gz
-rw-r----- 1 syslog adm 308627 2024-02-15 10:17:29.000000000 +0000 kern.log.4.gz
-rw-r----- 1 syslog adm 30080233 2024-11-03 19:30:01.122317630 +0000 syslog
-rw-r----- 1 syslog adm 18621882 2024-07-17 08:14:44.584848231 +0100 syslog.1
-rw-r----- 1 syslog adm 2133254 2024-05-19 10:41:45.000000000 +0100 syslog.2.gz
-rw-r----- 1 syslog adm 2823717 2024-04-11 08:41:08.000000000 +0100 syslog.3.gz
-rw-r----- 1 syslog adm 219532 2024-02-19 09:45:18.000000000 +0000 syslog.4.gz
-rw-r----- 1 syslog adm 324801 2024-02-15 10:17:30.000000000 +0000 syslog.5.gz
-rw-r----- 1 syslog adm 39404 2024-02-09 10:09:28.000000000 +0000 syslog.6.gz
-rw-r----- 1 syslog adm 40455 2024-02-08 10:18:28.000000000 +0000 syslog.7.gz
-rw-r----- 1 syslog adm 3349391 2024-11-03 19:15:54.093842183 +0000 ufw.log
-rw-r----- 1 syslog adm 1799969 2024-07-16 09:56:33.691038861 +0100 ufw.log.1
-rw-r----- 1 syslog adm 128488 2024-05-18 12:06:01.000000000 +0100 ufw.log.2.gz
-rw-r----- 1 syslog adm 219247 2024-04-10 09:52:17.000000000 +0100 ufw.log.3.gz
-rw-r----- 1 syslog adm 40975 2024-02-14 12:42:53.000000000 +0000 ufw.log.4.gz
Not sure if the result above is what I should be after, but seems informative.
And how about this:
The symptoms described here may have appeared because log files are not rotated properly
And how can the log files rotation be remedied?
You could try to do it manually first, and see if anything changes:
sudo journalctl --rotate --vacuum-time=1week
NB: note that the equal sign was missing in the --vacuum-time= option in the command you posted above.
I'll try, wish me luck :)
Note that it is very possible that nothing will happen.
The problem you are describing is related to Log File Viewer, which shows the content of files in /var/log, but not from /var/log/journal. Most of the /var/log content is in fact pulled by syslog daemons from /var/log/journal, so my assumption was that the problem could be happening either in both places, or only in /var/log.
Now if nothing happens when you try to rotate the journal (which is stored in /var/log/journal) it probably means that we can focus on what is going on in /var/log. As a wild, wild, guess, I would suspect that some job or some state may be interfering with the writing process: the computer being off at scheduled time, some backup running, or watnot.
Almost missed this! Glad I didn't, it's very informative, even if widens the search for solutions.
One thing I can say, is that there are no running backups, or jobs, or anything else that isn't initiated by the user. So, unless there is some unkown 'something else' running, this could be ignored.
Thank you Prospero, there is more to think about now.
Wondering what you would get if you ran:
systemctl status *timer | grep logrotate
Got this Prospero:
$ systemctl status *timer | grep logrotate
● logrotate.timer - Daily rotation of log files
Loaded: loaded (/lib/systemd/system/logrotate.timer; enabled; vendor preset: enabled)
Triggers: ● logrotate.service
Docs: man:logrotate(8)
man:logrotate.conf(5)
I also tried this (with info from --help, that could be used for 'time')
$ ls -l /var/log --time=status
total 95000
-rw-r--r-- 1 root root 0 Jul 17 08:14 alternatives.log
-rw-r--r-- 1 root root 7602 Jul 17 08:14 alternatives.log.1
-rw-r--r-- 1 root root 651 Jul 17 08:14 alternatives.log.10.gz
-rw-r--r-- 1 root root 591 Jul 17 08:14 alternatives.log.11.gz
-rw-r--r-- 1 root root 6960 Jul 17 08:14 alternatives.log.12.gz
-rw-r--r-- 1 root root 193 Jul 17 08:14 alternatives.log.2.gz
-rw-r--r-- 1 root root 641 Jul 17 08:14 alternatives.log.3.gz
-rw-r--r-- 1 root root 469 Jul 17 08:14 alternatives.log.4.gz
-rw-r--r-- 1 root root 469 Jul 17 08:14 alternatives.log.5.gz
-rw-r--r-- 1 root root 605 Jul 17 08:14 alternatives.log.6.gz
-rw-r--r-- 1 root root 550 Jul 17 08:14 alternatives.log.7.gz
-rw-r--r-- 1 root root 523 Jul 17 08:14 alternatives.log.8.gz
-rw-r--r-- 1 root root 730 Jul 17 08:14 alternatives.log.9.gz
drwxr-xr-x 2 root root 4096 Mar 6 2023 apparmor
drwxr-xr-x 2 root root 4096 Nov 3 14:25 apt
-rw-r----- 1 syslog adm 1112236 Nov 4 18:51 auth.log
-rw-r----- 1 syslog adm 684513 Jul 17 08:14 auth.log.1
-rw-r----- 1 syslog adm 28663 Jul 17 08:14 auth.log.2.gz
-rw-r----- 1 syslog adm 36018 Jul 17 08:14 auth.log.3.gz
-rw-r----- 1 syslog adm 5583 Jul 17 08:14 auth.log.4.gz
-rw------- 1 root root 773096 Nov 4 18:48 boot.log
-rw------- 1 root root 510401 Jul 17 08:14 boot.log.1
-rw------- 1 root root 364311 Jul 17 08:14 boot.log.2
-rw------- 1 root root 478349 Jul 17 08:14 boot.log.3
-rw------- 1 root root 36229 Jul 17 08:14 boot.log.4
-rw------- 1 root root 57023 Jul 17 08:14 boot.log.5
-rw------- 1 root root 6396 Jul 17 08:14 boot.log.6
-rw------- 1 root root 6962 Jul 17 08:14 boot.log.7
-rw-r--r-- 1 root root 66692 Mar 6 2023 bootstrap.log
-rw-rw---- 1 root utmp 3072 Oct 25 09:38 btmp
-rw-rw---- 1 root utmp 1920 Jul 17 08:14 btmp.1
drwxr-xr-x 2 root root 4096 Jul 17 08:14 cups
drwxr-xr-x 2 root root 4096 Mar 6 2023 dist-upgrade
-rw-r--r-- 1 root adm 71051 Nov 4 18:48 dmesg
-rw-r--r-- 1 root adm 71052 Nov 4 18:48 dmesg.0
-rw-r--r-- 1 root adm 18870 Nov 4 18:48 dmesg.1.gz
-rw-r--r-- 1 root adm 19266 Nov 4 18:48 dmesg.2.gz
-rw-r--r-- 1 root adm 19405 Nov 4 18:48 dmesg.3.gz
-rw-r--r-- 1 root adm 18903 Nov 4 18:48 dmesg.4.gz
-rw-r--r-- 1 root root 18984 Nov 3 14:25 dpkg.log
-rw-r--r-- 1 root root 149629 Jul 17 08:14 dpkg.log.1
-rw-r--r-- 1 root root 17991 Jul 17 08:14 dpkg.log.10.gz
-rw-r--r-- 1 root root 13639 Jul 17 08:14 dpkg.log.11.gz
-rw-r--r-- 1 root root 8474 Jul 17 08:14 dpkg.log.12.gz
-rw-r--r-- 1 root root 9310 Jul 17 08:14 dpkg.log.2.gz
-rw-r--r-- 1 root root 860 Jul 17 08:14 dpkg.log.3.gz
-rw-r--r-- 1 root root 16154 Jul 17 08:14 dpkg.log.4.gz
-rw-r--r-- 1 root root 10143 Jul 17 08:14 dpkg.log.5.gz
-rw-r--r-- 1 root root 11031 Jul 17 08:14 dpkg.log.6.gz
-rw-r--r-- 1 root root 13442 Jul 17 08:14 dpkg.log.7.gz
-rw-r--r-- 1 root root 6204 Jul 17 08:14 dpkg.log.8.gz
-rw-r--r-- 1 root root 7420 Jul 17 08:14 dpkg.log.9.gz
-rw-r--r-- 1 root root 32032 Mar 18 2023 faillog
-rw-r--r-- 1 root root 11338 May 31 15:17 fontconfig.log
-rw-r--r-- 1 root root 308394 Jun 18 2023 gufw.log
drwxr-xr-x 3 root root 4096 Mar 6 2023 hp
drwxr-xr-x 2 root root 4096 Mar 6 2023 installer
drwxr-sr-x+ 3 root systemd-journal 4096 Mar 6 2023 journal
-rw-r----- 1 syslog adm 16164989 Nov 4 18:52 kern.log
-rw-r----- 1 syslog adm 9895000 Jul 17 08:14 kern.log.1
-rw-r----- 1 syslog adm 1252836 Jul 17 08:14 kern.log.2.gz
-rw-r----- 1 syslog adm 1803742 Jul 17 08:14 kern.log.3.gz
-rw-r----- 1 syslog adm 308627 Jul 17 08:14 kern.log.4.gz
-rw-rw-r-- 1 root utmp 292292 Mar 18 2023 lastlog
drwxr-xr-x 2 root root 4096 Jul 17 08:14 lightdm
drwxr-xr-x 2 root root 4096 Mar 6 2023 openvpn
drwx------ 2 root root 4096 Mar 6 2023 private
drwxr-x--- 2 root adm 4096 Oct 29 2023 samba
drwx------ 2 speech-dispatcher root 4096 Mar 6 2023 speech-dispatcher
-rw-r----- 1 syslog adm 30482575 Nov 4 18:52 syslog
-rw-r----- 1 syslog adm 18621882 Jul 17 08:14 syslog.1
-rw-r----- 1 syslog adm 2133254 Jul 17 08:14 syslog.2.gz
-rw-r----- 1 syslog adm 2823717 Jul 17 08:14 syslog.3.gz
-rw-r----- 1 syslog adm 219532 Jul 17 08:14 syslog.4.gz
-rw-r----- 1 syslog adm 324801 Jul 17 08:14 syslog.5.gz
-rw-r----- 1 syslog adm 39404 Jul 17 08:14 syslog.6.gz
-rw-r----- 1 syslog adm 40455 Jul 17 08:14 syslog.7.gz
-rw-r----- 1 syslog adm 3359630 Nov 4 10:18 ufw.log
-rw-r----- 1 syslog adm 1799969 Jul 17 08:14 ufw.log.1
-rw-r----- 1 syslog adm 128488 Jul 17 08:14 ufw.log.2.gz
-rw-r----- 1 syslog adm 219247 Jul 17 08:14 ufw.log.3.gz
-rw-r----- 1 syslog adm 40975 Jul 17 08:14 ufw.log.4.gz
drwxr-x--- 2 root adm 4096 Jul 17 08:14 unattended-upgrades
-rw-rw-r-- 1 root utmp 1157376 Nov 4 18:49 wtmp
-rw-rw-r-- 1 root utmp 1053696 Dec 27 2023 wtmp.1
-rw-r--r-- 1 root root 237 Mar 6 2023 wvdialconf.log
-rw-r--r-- 1 root root 44487 Nov 4 18:49 Xorg.0.log
-rw-r--r-- 1 root root 45532 Nov 4 18:48 Xorg.0.log.old
It is not just Syslog, kern and others have the extra months of files.
Can you remember what you were doing on July 17 at 08:14? All logs in /var/log except the active log files and install logs froze at that time. The active logs have probably kept inflating ever since, they seem to have the expected size for normal logging activity for that amount of time.
You can always try running logrotate manually. It is supposed to have the same effect as journalctl --rotate, but applied to the logs stored in /var/log.
Can you remember what you were doing on July 17 at 08:14?
Very good question, Prospero, and analysis! I was wondering about that last night, if something was messed up when I was helping with updates and other things. I'll check and see what I can find.
You can always try running logrotate manually. It is supposed to have the same effect as journalctl --rotate, but applied to the logs stored in /var/log.
How do I run logrorate manually? I've never done anything like that.
You will find detailed information about logrotate in its manual page: $ man logrotate
For a summary of logrotate options, see: $ logrotate --help
You should be able to rotate your logs manually by running:$ sudo logrotate /etc/logrotate.conf
That is great Prospero. THANK YOU!!
I'll be checking the problem computer tomorrow, hope *July 17, 08:14* will find some answers.
THANK YOU Prospero!!!!!!!
Hurray for that:
$ sudo logrotate /etc/logrotate.conf
It worked! All logs have been rotated and it's all in order.
Good! I am glad you did not get ingested by the log monster, becoming a series of compressed bits in a black hole system.
By the way, you can get the full status of the systemd logrotate timer by using this simpler command:
$ systemctl status logrotate.timer
It will tell you when the next log rotation is supposed to be triggered.
Haha, that image feels so real!
More big thanks for that command too Prospero! It will come very handy to keep an eye on the monster, and check if that strong gravity pull has been tamed once and for all.
Indeed. I like to stick to the explicit syntax of the manual, but you created a discrepancy.
No luck Prospero :(
I think journalctl and syslog are different programs, syslog is old one. There some links
https://sematext.com/blog/journald-logging-tutorial/
https://www.fosslinux.com/111830/how-to-empty-or-clear-system-log-files-in-linux.htm
https://stackoverflow.com/questions/35638219/ubuntu-large-syslog-and-kern-log-files
You seems to have large syslog files. I don know how to safely clean them.
Thank you for the links grosbidepoilu!
I checked the stackoverflow (confusing!) and will look at the others in a mo.
-rw-r----- 1 syslog adm 30080233 2024-11-03 19:30:01.122317630 +0000 syslog
That is the largest file you listed, but it is only 30 MB large. As grosbidepoilu hinted, journalctl uses /var/log/journal. Here is a command to see the sizes of its files:
$ du -h /var/log/journal/
30 MB is already large for an active syslog file.
Also, syslog.1 was last modified on 2024-07-17. This is somewhat long ago for a daily rotation.
I agree Prospero, this doesn't look good at all and seems to have no obvious solution, at least for now... there are more replies to read (crossing fingers again).
The list of files is so very long, that it takes minutes upon minutes to load, and that makes it almost unusable.
I forgot to add the specifics, so here they are, hoping this might help - the computer is an TP X230 with Trisquel 10.
> Trisquel 10
This may explain why the ls command did not want to hear anything about --time=birth, which was a bit puzzling.
If the content of /home/someoneelse has been properly backed up, I would suggest a fresh install of Trisquel 11 Aramo.
He doesn't want to update yet, as I didn't manage to configure Trisquel 11 to match Trisquel 10.
It is the same for me, I messed up many installs and gave up for now.
Is there a solution for this problem in Trisquel 10?
I forgot to add on the specifics, the computer (TP X230) is Librebooted and probably has blobs. Could that have something to do with it?
Must add that I am not alone in wanting a system that doesn't network, that has no remote connection at all, a stand alone system.
When I hear this, I wonder why there isn't an option to kill all remote in one go. Or maybe there is a kill switch, but I haven't found it... would love to find it though!
By the way, when you love Trisquel as we do, we want badly to make it work for us too!