Root Partition Filled Up After Upgrade (6 > 7)
Hi everyone, I just upgraded my second laptop (Fujitsu Lifebook) from Toutatis to Belenos using the Update Manager. Now I am getting warning messages that my root partition is full, and I have no idea why that would be.
When I installed Toutatis, I set up a 16GB root partition, which I thought was plenty large enough. I put /home on a separate partition, and that seems to still be the case. The only extra applications I had installed were a few games 0 AD, FreeCiv, MineTest, and Egoboo which was removed during the upgrade (no longer in the repos, why is that?). AFAIK I had plenty of space in the root partition before the upgrade.
I'm guessing I could solve the problem by doing a clean install of Belenos. But does anyone have any idea what's happened, or what else I might be able to do to fix the problem?
You can see what takes space from the "Disk usage analyzer" (the command is 'baobab' if you do not want to search in the menus).
Two probable causes and solutions:
- You never remove the .deb packages and /var/cache/apt/archives/ eats up the space -> 'sudo apt-get clean' to remove all .deb files; 'sudo apt-get autoclean' to only keep those that relates to the latest versions;
- You kept many kernels (you only need the last one and, for safety, one older one you are sure it properly boots) -> remove the linux-image-*, linux-image-extra-* and linux-headers-* packages (for instance from the "Synaptic package manager" in the "System settings") that relate to versions of the kernel you do not need anymore (that can free close to 200MB per kernel).
I do run autoremove and autoclean from time to time. They were the first things I tried. That cleared 8KB, but 14.5 GB is still being used by something. I will try your second suggestion next, but this was a fresh install of Toutatis a couple of days ago, so I doubt old kernels would be taking up more than a few MB.
EDIT: Checked in Synaptic. Only one version of each of the kernel packages you mention is installed, so it's not that.
EDIT: Baobab reports:
/var 69% (9.8GB)
/usr 28% (4.3GB)
/lib 1.9% (290.6MB)
That accounts for 98.8%, so everything else is pretty negligable.
Within /var:
97.2% is in /log (9.5GB), but when I try to drill down further, there's nothing in any subdirectory of /log that accounts for even a fraction of that.
Within /usr:
61.5% is in /share (2.7GB)
35% is in /lib (1.4GB)
2.6% is in /bin (106.9MB)
Most of what's in /lib is in /modules (88.8% of /lib), and that's less than 300MB.
Is there a command for clear out the contents of /log?
Are you using btrfs? One time I tried it on my root partition and it filled up quickly due to its copy on write nature. It was a little while back though, so maybe the space issue was fixed in later kernels.
Nope. I'm using ext4 on both root and /home partitions, but only because ext is what I'm familiar with, and 4 is the latest version and has been out for a while. Are there better choices?
Well, you can use ext2 on boot partition as it doesn't use journaling.
The TL;DR of my reply to Magic Banana is that I seem to have almost 10GB of logs in /var/log. Is it safe for me to just go into that folder and delete everything in there that's not a folder? Or is there is a special command for safely purging logs?
Also, the size of the logs seems to be growing constantly. I cleared some space with auto-remove and now my root partition is full again. I suspect this may be a bug. I will be able to confirm this once I can remove the logs, and see if they start to fill up the partition again. If so, I will file a bug report.
Deleting logs may work but it might be better to determine which program is generating the logs and address that. Either start doing log rotation, have it log nothing (or less stuff if it turns out the logs are overly verbose and you don't need that) or whatever...
I had a look in /var/log using ls -l. It appears the three files which are taking up most of the space are syslog (6.3GB), dpkg.log (3.4GB), and kern.log (3.2GB). This seems like a ridiculous size for any kind of log. The system also has syslog.1 (294.9KB) and kern.log.1 (804.8KB), which are much smaller, but still very large for text files.
Again, this was a fresh install of Toutatis from LiveCD about 3 days ago, then upgraded to Belenos using Update Manager, and I haven't manually instructed it to log anything. Keeping all this in mind, I'm tending towards thinking that a 16GB root partition filling up with logs is a sign of a bug.
EDIT: I deleted those three files from /var/log, and rebooted. The syslog.1 and kern.log.1 files are still there, but the system has created a new syslog (already above 200MB) and kern.log (already above 100MB). I will file a bug.
EDIT: done: https://trisquel.info/en/issues/15696
This happened to me once on a computer where I hadn't yet removed an Intel wireless card non-supported by linux-libre. Do you have any hardware parts that don't work in the free world on that machine?
You're right, it may be that the system is filling the logs with errors of that kind --- mmmm ----
Yes, there is a device that doesn't yet work under free code, "Network controller: Intel Corporation PRO/Wireless 3945ABG", as discussed here:
https://trisquel.info/en/forum/fresh-toutatis-install-fujitsu-lifebook-wireless-internet-not-working
Does this mean that the cancerous logs are nothing to do with the fact I upgraded from 6 > 7, and that a fresh install of 7 won't fix it? I was hoping it might :(
(BTW Sorry for not responding to the questions on the bug report I filed, will get onto that today)
EDIT: done: https://trisquel.info/en/issues/15696
logrotate is used by the system to rotate logs, so you have two choices. You can either incorporate the rotation of these app logs into the systems rotations or setup your own and either run them manually or from the root user's crontab (Assuming the Rails app is run as root given it's directory is /root/)
To setup a logrotation within the system's pre-existing ones simply add a new file to the directory /etc/logrotate.d. Call it railsapp.conf. I'd use the other examples there to construct it. Also confrere with the logrotate man page.
User rotation
If you want to run your own instance of logrotate you only have to provide it with command line switches to do so.
First make a copy of:
/etc/logrotate.conf /root/rails_logrotate.conf
Edit the file so that it has the log rotation configured the way you want (i.e. keep all logs, rotate weekly, etc.)
Run it
# 1st time
$ logrotate -d -f -s $HOME/my_logrotate.state logrotate.conf
# afterwards
$ logrotate -d -s $HOME/my_logrotate.state logrotate.conf
If things look alright you can re-run these commands without the -d switch. This is for debugging purposes only and won't actually do any of the tasks, merely show you what it WOULD do.
$ logrotate -s $HOME/my_logrotate.state logrotate.conf
You could also use the -v switch to make it verbose, similar to the output seen when using the -d switch.
Example
Start with this log file.
$ dd if=/dev/zero of=afile bs=1k count=10k
10240+0 records in
10240+0 records out
10485760 bytes (10 MB) copied, 0.0702393 s, 149 MB/s
$ ll afile
-rw-rw-r-- 1 menomore menomore 10485760 Oct 26 14:37 afile
$ touch -t 201307010101 afile
$ ll afile
-rw-rw-r-- 1 menomore menomore 10485760 Oct 26 01:01 afile
Now run logrotate
$ logrotate -v -f -s $HOME/my_logrotate.state logrotate.conf
reading config file logrotate.conf
reading config info for /home/menomore/afile
Handling 1 logs
rotating pattern: /home/menomore/afile forced from command line (1 rotations)
empty log files are rotated, old logs are removed
considering log /home/menomore/afile
log needs rotating
rotating log /home/menomore/afile, log->rotateCount is 1
dateext suffix '-20130806'
glob pattern '-[0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9]'
glob finding old rotated logs failed
renaming /home/menomore/afile to /home/menomore/afile-20130806
creating new /home/menomore/afile mode = 0664 uid = 500 gid = 501
Check the results
$ ll afile*
-rw-rw-r-- 1 menomore menomore 0 Oct 26 14:40 afile
-rw-rw-r-- 1 menomore menomore 10485760 Oct 26 01:01 afile-20130806
Weekly;
To make this run every Sunday you could create the following crontab entry for the root user.
$ crontab -e
Add the following lines:
# Example of job definition:
# .---------------- minute (0 - 59)
# | .------------- hour (0 - 23)
# | | .---------- day of month (1 - 31)
# | | | .------- month (1 - 12) OR jan,feb,mar,apr ...
# | | | | .---- day of week (0 - 6) (Sunday=0 or 7) OR sun,mon,tue,wed,thu,fri,sat
# | | | | |
# * * * * * user-name command to be executed
0 0 * * sun logrotate -v -f -s $HOME/my_logrotate.state $HOME/logrotate.conf
Then save the above.
You can also use these types of shortcuts instead of specifying the actual days, mins, secs, etc.
string meaning
------ -------
@reboot Run once, at startup.
@yearly Run once a year, "0 0 1 1 *".
@annually (same as @yearly)
@monthly Run once a month, "0 0 1 * *".
@weekly Run once a week, "0 0 * * 0".
@daily Run once a day, "0 0 * * *".
@midnight (same as @daily)
@hourly Run once an hour, "0 * * * *".
I did this a while ago, I thiink my way COULD be wrong. But we all figure out how to, because what I googled confused me, so I needed to figure it out on my own. You can do it your own way, I guess.
If you like, a reference:
http://www.thegeekstuff.com/2010/07/logrotate-examples/
Thanks for the info MeNoMore, but I'm thoroughly lost. Could you maybe try explaining this again, like I'm 5 years old?
You've read the rest of this thread, yes? You understand that we're discussing a laptop, not a server, yes? Why would I want to use logrotate, and how will it help me solve my laptop's log cancer problem?
> You've read the rest of this thread, yes? You understand that we're
> discussing a laptop, not a server, yes?
Why the viciously sarcastic tone? You asked for help, the above post gave you
some. Just because you see some command-line stuff that is a step more
complicated and requires a bit of understanding as opposed to 'click this,
click that' doesn't mean you need to just dismiss it out of hand as being 'for
sysadmins, not mere mortals'.
> Why would I want to use logrotate,
> and how will it help me solve my laptop's log cancer problem?
If you'd paid any attention to the post, instead of simply crying out 'tl;dr',
you'd see that in some cases partitions can fill up due to logs filling up out
of control. Logrotate can rotate logs out and save disk space, preventing that
sort of thing from happening.
Thank you.
I was simply asking for clarification, so as not to waste the other person's time or my own going down a blind alley. MeNoMore gave me some very helpful clarification, for which I said thank you. All of this happened well before you posted this unhelpful attack comment, and your slanted perception of my "tone" is not my responsibility.
Sorry. I went back and read your post properly, and have no idea by the clear
light of day why I interpreted it as an attack last night. Sorry again.
We are all capable of douchebaggery of the highest order, it seems.
Apology accepted :) One major downside of text communication is the ability to read exactly the same words in a million different ways.
>> We are all capable of douchebaggery of the highest order, it seems. <<
I'm certainly not immune :P
> Apology accepted :)
... Captain Needa ;)
I am a translator!
> You've read the rest of this thread, yes? You understand that we're
> discussing a laptop, not a server, yes?
moxalt: I would not define that "viciously sarcastic". Just inappropriately sarcastic. The tone is certainly sarcastic. There is no doubt about it. It is also inappropriately sarcastic being that it was directed to someone who tried to help.
mox already apologized & everybody kissed and hugged
Ooohhh ssoooo nice from you!!! Moxy, my poor lil thang!
>> The tone is certainly sarcastic. There is no doubt about it. <<
No. You are *interpreting* the tone as being sarcastic, which is something that is about you, not me. The very definition of sarcasm is that it uses words that appear to mean one thing, combined with a context (usually a tone of *voice*) that confirms that what is really meant is the opposite of those words. Even in a spoken conversation, especially with someone you don't know well, it's possible to mistake a genuine statements for sarcasm, and vice versa. That you think it's possible to make a rock solid distinction between the two in a text conversation is quite frankly laughable, particularly because given a 50/50 chance of getting it right by dumb luck, you still got it wrong :)
I am a translator!
> You've read the rest of this thread, yes? You understand that we're
> discussing a laptop, not a server, yes?
The reiteration of a question (you read the rest of this thread/You understand that we're discussing a laptop, not a server) followed by the reiteration of the questioning yes (,yes?)
is without any doubt and objectively sarcastic. It has nothing to do with *interpreting*. You were intentionally sarcastic and we both know that was your intention mr. strypey..
I think this is a cultural difference. Perhaps this particular sentence form (question, yes? question, yes?) is always used sarcastically where you're from. That's not true here.
New Zealand English is influenced by the indigenous Māori language, of which I am a basic speaker. In Māori, there are no special sentence forms that distinguish a question from a statement. The expression "nē rā" (or just "nē"), meaning something like "is that right?", or "truly?", is added at the end of a sentence to seek confirmation from the listener, turning what would otherwise be a statement into a question. My use of a statement, followed by "yes?", follows this habit.
>> It has nothing to do with *interpreting*. You were intentionally sarcastic and we both know that was your intention mr. strypey.<<
You can believe whatever you like, but I know what I intended, therefore I know you are wrong. MeNoMore has taken no offence, and to be honest, I'm not sure why it's so important to you to take it on their behalf. It's easy to read "mr. strypey" as sarcastic and provocative, but I simply choose not to take offence. Try it, it's easy :)
What is logrotate?
logrotate is a program used to rotate logs. The system usually runs logrotate once a day, and when it runs it checks rules that can be customized on a per-directory or per-log basis.
“Log rotation” refers to the practice of archiving an application’s current log, starting a fresh log, and deleting older logs. Because a “log” is a file where an application stores information that might be useful to an administrator or developer - what it’s been doing, what errors it’s run into, that sort of thing. So logs are good, you just usually don’t want to keep a ton of them around. That’s where logrotate comes in.
The importance of log rotation
Logs are wonderful things when you want to track usage or troubleshoot an application. Unfortunately the more information that gets logged, the more disk space the log uses. Over time it can really add up.
A log left unrotated can grow to a pretty unwieldy size. Running out of disk space because of a giant log is a problem of course, but a huge log file can also slow down the process of resizing or backing up. Another practical consideration is that it’s hard to look for a particular event if you have a million log entries to skim through. So on the whole it’s a good idea to keep log files down to a manageable size, and to prune them when they get too old to be of much use.
Fortunately logrotate makes log rotation easy.
Okey okey... we got it. The Almighty LogRotate is our budha and you love it. pppppp
Just kidding dude, thanks for the cool stuff. It'll come in handle later on my future.
I am a translator!
the dude or girl spent a considerable amount of time to write the guide to help and you joke about it..
https://taikonenfea.files.wordpress.com/2013/07/shame-award-112440855821-1.jpg
cheers :)
http://trisquel.info/en/forum/five-nights-freddies-trisquel#comment-76431
Also, to see how the trisqueltrollsquad members can make jokes I can link this
http://trisquel.info/en/forum/five-nights-freddies-trisquel#comment-76208
Cheers
Why the secondary e-mail address? someonedidnotlikeitsothisishowitlooksnow as
opposed to cooloutac? I've been wondering about that for a bit.
Secondary? If you'r asking why I put someonedidnotlikeitsothisishowitlooksnow as e-mail address It's because you (moxalt) had complained about it.
"as opposed to cooloutac?" I can't help u here as I don't have any clue on what cooloutac means. Cool out activation? Cool! Out of Alternate Current? I don't c a correlation with my address.
On Thu, 29 Oct 2015 19:57:55 +0100 (CET),
name at domain wrote:
> Secondary? If you'r asking why I put
> someonedidnotlikeitsothisishowitlooksnow as e-mail address It's because you
> (moxalt) complain about it.
Calm down. I'm not out to get you.
> "as opposed to cooloutac?" I can't help u here as I don't have any clue on
> what cooloutac means. Cool out activation? Cool! Out of Alternate Current? I
> don't a correlation with my address.
So you're not actually one of cooloutac's alts? I apologise. Equally
terrible grammar, spelling, conspiracy theories, the belief that your
neighbours are more dangerous than the intelligence apparatus, and constant
reference to the 'trisqueltrollsquad' made you indistinguishable from
cooloutac. I thought you were one of his sock puppets.
That was confusing. I post by e-mail, and rarely (if ever) can be bothered to
actually log in to the web site itself. So I know people more by their e-mail
addresses than their forum nicks- I think of SuperTramp as 'greatgnu', Magic
Banana as 'lcerf', etc. Originally cooloutac posted (and still does?) with an
address beginning in the same nick- cooloutac@something. Then came you
(antiesnob) posting with an e-mail address actually beginning 'antiesnob'. So
when this strange name at domain' showed
up, I thought (due to the numerous similarities) it was one of cooloutac's alts.
I am a translator!
moxalt - you are wrong on this one. antiesnob has his strange way to joke and is not very good with English, but he's a decent dude overall and he's not that other major troll known as Coolout. I can guarantee you that. I read a lot of antiesnob's post both here and in the Spanish forum. I'm quite good with linguistic style (recognizing the patterns and the similarities) and I bet my ugly ass those two guys have nothing to do with each other. :)
"is not very good with English"
I am very good with English, on English, above English and on top of English but not below coz I don't feel comfortably in that position. LMAO
"he's a decent dude overall"
Yes. Except that time three years ago that I lost a bet to GNUser and I had to post a picture of my dead-alive dick. Here's the thread
http://trisquel.info/en/forum/brigantias-livecd-gparted-screwed-my-800gb-ext4-home-d
From now on Sup3rTramp is my lawyer-in-law
I'm happy to see my family reunited here. Moxalt my daddy, Sup3rTram my lawyer-in-law and my step-mommy Heather. The very first time we met was with my first distro she was there!! The Linux Header!! Luv u Header!
> I'm happy to see my family reunited here. Moxalt my daddy, Sup3rTram my
> lawyer-in-law and my step-mommy Heather. The very first time we met was with
> my first distro she was there!! The Linux Header!! Luv u Header!
Oh. I am sincerely sorry. I now correctly recognise you as the YOU HAVE BEEN
TROLLED person.
I wonder how is that u didn't confuse me with Jodiendo instead of that guy that seems to have ideas totally opposed to mine.
I am a translator!
If I am your lawayer-in-law I fear about your future son..
:)
Well ,then... DO SOMETHING!!! Why do you think I'm payiinngg yoouuu!!! LOL I'm paying you to keep out of prision!!! LOL
"Calm down. I'm not out to get you."
I was calmed down and I still am. You had complained about my email address and then I changed it ironically. This is the third time I've got to explain this to you.
"conspiracy theories"
I didn't said such a thing. The trisqueltrollsquad is a name for a set of exclusive users. There's no such a thing as a conspiracy, organization, squad, team nor a nation named trisqueltrollsquad. That was made up by another member of the trisqueltrollsquad named MagicBanana and another one I forgot the name. I haven't said there's a conspiracy against me not even a persecution against me (opposed as what you said in another post that I think I'm being persecuted).
How does it comes that I talk about conspiracy theories and you are talking I am someone else?
"the belief that your neighbours are more dangerous than the intelligence apparatus"
I haven't made any reference to it. Now I can tell you that in my neighborhood were 4 Inn Tel ngy Agnts. One of them passed away and was actually retired, another one had his services to another "state" removed (it's not another state but it is like saying he was removed from California to Texas). The other two are operative. I should say that there were another two agents on the University I went. From the 6 I have mentioned only the latter 2 were "chasing" me. They do not operate in the same neighborhood they live. That's how it works here I don't know why.
For your information me and my family were persecuted by the police during years and a few of them lived in my neighborhood. Now there's only two that still live here. All that persecution stopped about 3 and a half years ago.
"constant reference to the 'trisqueltrollsquad' made you indistinguishable from cooloutac."
I'm pretty sure nobody called that name to this set of users. I used to call them 'TheEternalOffTopickers', if that makes you feel better. I0ve changed that named after the introducing of the useless spammerbegone-tool as I stopped working on the wiki for the same reason because I'm truly offended with that crap and the ability to censor comments the trisqueltrollsquad don't like.
Sincerely yours,
Your beloved sweetums
ps: how hard you try not to get on with me is astonishing.
The trisqueltrollsquad is a name for a set of exclusive users.
In what way are those users "exclusive"? I am interesting to know since, according to you, I am a "member" of that "exclusive" club but I ignore that I can enjoy any kind of privileges!
I used to call them 'TheEternalOffTopickers', if that makes you feel better.
Kind of ironic given the post you have written.
I0ve changed that named after the introducing of the useless spammerbegone-tool as I stopped working on the wiki for the same reason because I'm truly offended with that crap and the ability to censor comments the trisqueltrollsquad don't like.
Every user of the forum can click the +1 and the -1 buttons. I sometimes (but rarely) click those buttons. In fact, I have just clicked the +1 button on https://trisquel.info/forum/root-partition-filled-after-upgrade-6-7#comment-82660 (because it was at -1 for no good reason) and the -1 button on https://trisquel.info/forum/root-partition-filled-after-upgrade-6-7#comment-82671 (because I see no reason to believe strypey is lying and, even if he was, because I am against witch-hunting).
The +1 (resp. -1) button is worth +1 (resp. -1) whoever the user. There is no "exclusive set of users" that would have votes with bigger weights.
I have never said the trisqueltrollsquad are exclusive users because of the useless spammersbegonetool.
All replies to an offtopic comment are offtopic. And guess what? I was not the one making the first offtopic comment!
-2 means 2 idiots clicked...
None demonstrated/quoted me saying what this user said I said...
And also there are 2 idiots who said that replying to off topics are on topic messages.
Once again we can see how good a voting system is!! It is designed so idiots can show themselves (but without saying their names because they don't want to be noticed as idiots).
Thanks MeNoMore. That's cleared up a lot of the questions I had :)
Don't be deceived by the trisqueltrollsquad MeNoMore; I've found your explanation useful as I stated after the joke I make. I urge u 2 read it again, please. Don't be deceived:
"Just kidding dude, thanks for the cool stuff. It'll come in handle later on my future." (I clarify that there's no sarcasm on it)
Placing /var/log in its own partition or volume can help to mitigate the run-away disk usage. If you use audit, then having /var/log/audit on its own partition or volume helps to assure the audit logs are not affected.
tail -f /var/log/syslog will scroll the syslog (until it rotates) and let you see a little more of what's happening. If you do not want to do the logrotate.
I'd start by tailling the files and see what has most recently been writing to them, that might be enough of a clue. And then you can just echo blacklist the syslog.
Ae, the unsupported wireless chipset is definitely the prime suspect at this point. Since there is a lot of unsupported wireless hardware about, I think a and a detailed tutorial, and maybe prominent warning somewhere about this problem, would definitely be worthwhile.
At least you folks have saved me the time to resinstall Belenos, and the hair-tearing frustration when I realised it didn't help, so thanks for that :)
Do you think using the hardware switch to turn off the wireless subsystem would help?
Yes, I think that's what I did on my T60 as a first step before installing libreboot. A better solution is to actually remove the card though. You know, I suggested to you that this might be the problem several weeks ago. Could you please test and confirm/deny this now so that your issue can be closed. Then you can write the warning/tutorial on the wiki.