I’m still a fairly new Linux-user (on Tuxedo OS), and I just ran into an issue that is new to me. If I try to update my system, either via command line or Discover, the apt update command fails. This is the output:
E: Could not get lock /var/lib/apt/lists/lock. It is held by process 1635 (apt-get)
N: Be aware that removing the lock file is not a solution and may break your system.
E: Unable to lock directory /var/lib/apt/lists/
Process 1635 is apt-get update run by root, and persists through restart. I am tempted to try to kill it (kill 1635), but I’m not sure if anything could break from that, so I thought I’d try to ask for help first before I do something stupid.
EDIT:
I have managed to update my system by killing the process, which releases the lock, and then going on to do normal sudo apt update and sudo apt upgrade. For the sake of troubleshooting, I tried to add back my third-party repos one by one, and none of them caused any problem. However, when rebooting the same issue as described above happens again. Software updates is set to “Manually” in the System settings.
In addition, everytime I ran sudo apt upgrade, at the end some update related to initramfs fails. My disk is encrypted using cryptsetup, and as I’ve come to understand, I should be very careful doing anything related to initramfs when that is the case. Here is the output:
Processing triggers for initramfs-tools (0.140ubuntu13.2) ...
update-initramfs: Generating /boot/initrd.img-6.2.0-10018-tuxedo
I: The initramfs will attempt to resume from /dev/dm-2
I: (/dev/mapper/system-swap)
I: Set the RESUME variable to override this.
zstd: error 25 : Write error : No space left on device (cannot write compressed block)
E: mkinitramfs failure zstd -q -1 -T0 25
update-initramfs: failed for /boot/initrd.img-6.2.0-10018-tuxedo with 1.
dpkg: error processing package initramfs-tools (--configure):
installed initramfs-tools package post-installation script subprocess returned error exit status 1
Errors were encountered while processing:
initramfs-tools
E: Sub-process /usr/bin/dpkg returned an error code (1)
EDIT 2:
The issue seems to have been narrowed down to a failure of Tuxedo’s driver configuration service that runs at boot. It is this process that calls apt-get (and something I should’ve seen earlier…), and systemctl status reveals some errors:
aug. 08 15:33:56 laptop systemd[1]: Starting Tomte-daemon, finishes tasks that could not be accomplished before...
aug. 08 15:34:06 laptop tuxedo-tomte[1393]: no network found!! some fixes might not be applied correctly
aug. 08 15:34:06 laptop tuxedo-tomte[1393]: systemctlCmd: systemd-run --on-active="30sec" tuxedo-tomte configure all >/dev/null 2>&1
I really appreciate the help from everyone so far. It’s a good experience asking for help here, and I’ve learned a lot from your answers. Makes being a Linux newbie a lot easier. So thank you :)
Since this seems to be a very specific issue related to Tuxedo’s own services, I will contact their support to get their input on what to do next.
My first guess is unattended-upgrades is running, especially if this is shortly after booting. As others have said, ps aux | grep apt will tell you what’s running. If it’s holding up all the time there might be something wrong with apt causing the update to hang.
Yeah, it happens straight after boot, but it never resolves itself. And the process exists after reboot as well, so it seems that it starts running apt-get update and gets stuck / hangs.
Definitely sounds like auto-update if it’s respawning itself on every boot. The fact that it never exits is weird though; have you added any third party repos? What’s in your apt sources.list file(s)?
My
sources.listfile is pretty clean, with 3 https-sources for the Tuxedo OS mirror repos of the original Ubuntu ones. In thesources.list.ddirectory, there are some that have been added by me, such as for Signal, Librewolf and VS Codium. In total there are 11 files in here, each with one additional source. All except one ishttps, and the last one ismirror+file. In the process tree for apt-get, there are 13 subprocesses, while there are 14 sources in total (11+3). Could it be that it hangs on the last one here?EDIT: Would this be a viable way to troubleshoot? I backup the sources and just replace them with a blank
sources.listfile and an emptysources.list.ddirectory. If that works, I add the repos back one at the time and see which one that fails. Or could I run into unintended trouble if I remove the main repos, even for a short time? I would think that it just wouldn’t find anything and just be happy there are no updates.I don’t think that’s the problem, and I wouldn’t really anything related to
sources.listor anything insources.list.d/.(*)It does seem like an auto update thing, so got some quick suggestions.
Since you mentioned Discover, I’m guessing you’re on KDE Plasma, which is great because it’s what I use and know immediately where to go.
Go into the main settings and way at the bottom left after all the other sections, there should be a section labeled “Software Update”. The only thing you need to care about is the first option, which will be “Update software: Manually/Automatically”.
If “Automatically” is checked, try disabling it and then reboot. If that doesn’t, work, there’s a more useful GUI tool, but for some reason it doesn’t show up when searching the application menu (or at least doesn’t for me).
Quick command line thing here:
sudo software-properties-kdeThis has a bunch of options which I won’t get into, but in the “Updates” tab, you’ll also see “Automatic updates” with a few more options than the one in Settings. I’d enable “Only notify about available updates” if everything else is enabled.
You can disable “Check for updates” entirely, but I wouldn’t recommend it. You can change that to “Weekly” or something, so that it isn’t constantly checking, but I don’t know if it starts up on every boot, so might be useless to change that specifically.
After rebooting, take a look at what the file
/etc/apt/apt.conf.d/20auto-upgradessays.It shouldn’t be too long, but depending on what the values might be for
Download-Upgradable-PackagesandUnattended-Upgrades, that could be the source of the problem.Everything in that file should now have a value of “0” except for the first line (
Update-Package-Listswhich will have a value of “1”).(*) Theoretically, your idea about sources lists could maybe be an issue, but I’m not sure. I don’t know if it’s possible that
apt-getcould be in some kind of loop if there’s a mismatch or duplicates with your repo sources. I’d assume it would error out, but maybe it’s just being weird.For now, I’d leave that all alone. If none of the above works, I can give some instructions on dealing with third party repos, but I don’t want to make this comment any longer than it is already.
Auto update itself isn’t the root problem. The problem is that apt update is hanging and never finishing. It just happens to be getting called automatically as part of an auto update system, but the root issue would still persist even if OP disables auto updates.
When apt update fails to complete, it’s almost always because of a broken repo somewhere; hence my question about sources.list.
You may be right.
I’m wondering why
apt-getisn’t erroring out, though. Usually after enough tries, it just quits, at least in my experience.Probably best bet, then, is to remove and re-add the third party repos.
Ok, so now I have managed to update my system by killing the process, which releases the lock, and then going on to do normal
sudo apt updateandsudo apt upgrade. For the sake of troubleshooting, I tried to add back my third-party repos one by one, and none of them caused any problem.However, when rebooting, the same problem happens again. In system settings, auto update was already set to “Manually” and offline updates is unchecked. I have not made any modifications to this. I did not have
software-properties-kdeinstalled, and it was also not available by runningsudo apt install software-properties-kde. It suggested onlysoftware-properties-qtinstead. So I could not check those settings, but in the20auto-upgradesfile,Update-Package-ListsandUnattended-Upgradesare the only lines present, and both are set to 1.In addition, everytime I ran
sudo apt upgrade, at the end some update related toinitramfsfails. My disk is encrypted using cryptsetup, and as I’ve come to understand, I should be very careful doing anything related toinitramfswhen that is the case. Here is the output:Processing triggers for initramfs-tools (0.140ubuntu13.2) ... update-initramfs: Generating /boot/initrd.img-6.2.0-10018-tuxedo I: The initramfs will attempt to resume from /dev/dm-2 I: (/dev/mapper/system-swap) I: Set the RESUME variable to override this. zstd: error 25 : Write error : No space left on device (cannot write compressed block) E: mkinitramfs failure zstd -q -1 -T0 25 update-initramfs: failed for /boot/initrd.img-6.2.0-10018-tuxedo with 1. dpkg: error processing package initramfs-tools (--configure): installed initramfs-tools package post-installation script subprocess returned error exit status 1 Errors were encountered while processing: initramfs-tools E: Sub-process /usr/bin/dpkg returned an error code (1)Since this is buried quite a long way down into a thread, I will now also update the main post to include this update.
zstd: error 25 : Write error : No space left on device (cannot write compressed block)
What’s the output of
df -H?Also, this sounds like it’s installing initramfs, which is normally only done when first installing the OS; can we get a list of the packages it’s trying to install/upgrade?
This shows up when running
sudo apt updateafter having killed the process that is holding the lock. Where would I find out which packages it is trying to install or upgrade? It does sayupdate-initramfsin the second line there, so I think it’s trying to update?Relevant output from
df -H:/dev/nvme0n1p1 1,1G 945M 24M 98% /boot /dev/nvme0n1p2 536M 9,3M 527M 2% /boot/efi
I’d leave the main sources.list alone, but temporarily move all of the files out of sources.list.d and see if that fixes it.
As added as an edit in the original post, I managed to update my system by killing the process, did an upgrade with only the original sources.list and then added back the third-party repos one-by-one. None of the third party repos caused any problem when adding them back. Problem persists upon reboot. There seems to be an issue with updating
initramfsat the end of everysudo apt upgrade, but that shouldn’t run on boot, so I don’t think that can be the cause?
I haven’t used Tuxedo, but on apt-based distros it’s pretty common for an auto-update daemon of some kind to run in the background on startup to either download updates, or at least download package metadata so some UI component can start nagging you to install the updates that are available.
If you wait a few minutes, the download should complete and you can do what you want. You can probably get away with killing it, especially if you use a gentle signal like HUP. I wouldn’t risk it though… if you corrupt your package metadata or worse… and actual important package… it can be a significant hassle to clean up the mess. And the cost of waiting 30s-5m and trying again is so low it’s hard to beat that as an approach.
If its happening a ton you can probably find and disable the auto-update thing but I don’t know what it would be on Tuxedo.
It persists even after several hours. Never happened before, and I’ve been using the system for about 6 months now. I posted the output from the process tree below, but I’m not sure if that makes anything any clearer. It’s as if it is not receiving any data and just waiting for the servers to respond?
Check if the process is actually running
‘ps aux | grep 1635’
(You should see a line output)
More likely its not running and crashed without cleaning up the lock file, you might want to figure out why from logs and whatnot but its ok also to remove that file and try again
(You should see a line output)
Just a reminder that the line you see shouldn’t be the grep line. You can use ‘ps aux | grep 1635 | grep -v grep’
It’s showing up in the list with a bunch of sub-processes that looks like https-requests (I posted the output below). Which logs could I check to learn more about what’s happened?
When you say remove the file, do you mean the lock file?
killing the apt update shouldn’t break anything since its not installing any packages it’s just updating your repos. i would have just killed the pid 1635…also not sure if it would break something but i doubt it…did you dmesg to see what else info you might be able to find out?
Would there not be a risk of corrupting some of the repo files and dependencies lists by just killing it?
I have checked dmesg after your suggestion, but I did not see anything that tipped me off to what might be wrong. Is there anything in particular I should be looking for?
I would recommend trying out the process tree
Use
ps auxf | lessAnd then / to search for the process. See what child processes are running from that aptget. I wouldn’t expect much but that should give you an idea about the safety of killing it
Here is the output from running that command:
root 1635 0.0 0.0 22208 8928 ? S aug.06 0:00 \_ apt-get update _apt 1654 0.0 0.0 27528 9600 ? S aug.06 0:00 \_ /usr/lib/apt/methods/https _apt 1655 0.0 0.0 27528 9600 ? S aug.06 0:00 \_ /usr/lib/apt/methods/https _apt 1656 0.0 0.0 27528 9600 ? S aug.06 0:00 \_ /usr/lib/apt/methods/https _apt 1657 0.0 0.0 27528 9760 ? S aug.06 0:00 \_ /usr/lib/apt/methods/https _apt 1658 0.0 0.0 27528 9760 ? S aug.06 0:00 \_ /usr/lib/apt/methods/https _apt 1659 0.0 0.0 27528 9760 ? S aug.06 0:00 \_ /usr/lib/apt/methods/https _apt 1660 0.0 0.0 27528 9760 ? S aug.06 0:00 \_ /usr/lib/apt/methods/https _apt 1661 0.0 0.0 27528 9600 ? S aug.06 0:00 \_ /usr/lib/apt/methods/https _apt 1662 0.0 0.0 20908 6880 ? S aug.06 0:00 \_ /usr/lib/apt/methods/mirror+file _apt 1663 0.0 0.0 27528 9760 ? S aug.06 0:00 \_ /usr/lib/apt/methods/https _apt 1664 0.0 0.0 27528 9920 ? S aug.06 0:00 \_ /usr/lib/apt/methods/https _apt 1667 0.0 0.0 27532 10080 ? S aug.06 0:00 \_ /usr/lib/apt/methods/https _apt 1669 0.0 0.0 20864 7680 ? S aug.06 0:00 \_ /usr/lib/apt/methods/fileNice, so here’s your next step. Do a Google search and see what exactly that file is. Find out it’s purpose and that’ll give you an idea what’s going on, which then allows you to determine if it’s safe to kill or not.
I already looked it up, so I want you to do the same and let me know what you find out and what you think they are being used for.
As far as I can tell, these are the methods apt uses to get information from the repositories that is listed within
sources.listand within thesources.list.ddirectory. The number of subprocesses almost matches the number of sources there - in reality there are 14 listed, not 13 as is seen in thepsoutput. I can find one entry that starts withmirror+file, but otherwise there are 13httpsentries. So that last line I am not sure what is doing.Anyways, it seems to me that it gets stuck somewhere updating the repositories list. Right now, I’m stuck with three questions:
- I’m still unsure as to whether it would be safe to kill the process, as I could imagine that having a corrupted depencies files could be really bad?
- Also, would killing the process automatically release the lock, or would I need to remove that myself after?
- Is there any reason to believe that this would even work, seeing as this happens everytime on boot. I imagine that if I kill the process, delete the lock and try to run
sudo apt updateI just end up the same place again.
So my theory on what’s going on is this.
I suspect that this is an automatically running update to try to help keep your packages up to date, and I think it’s getting stuck on a source configured in your /etc/apt/sources.list I’m willing to bet it’s likely a source configured to pull from a “CD” which is used during installation and they forgot to disable that one.
You should be able to stop it, it’ll still be locked but you’ll need find the lock file (I forgot where it’s configured) and just remove it with a simple rm, you’ll probably need to sudo the rm though.
So my order of operations would be, kill the process, try to rerun the apt update and see if that tells you which repo it’s getting stuck on.
I managed to kill the process, and the lock was released with the operation, allowing me to upgrade my system. After this, I moved all external sources out of sources.list.d and upgraded my system, and continued to add them back one by one to see if any of them triggered an error. None of them did.
When I reboot the system, the problem is back. I’ve added some more details in the original post.
I’m guessing Tuxedo’s distro includes a service to automatically update the system or your repos on boot. It seems like the service is failing. I would take a look through your enabled services to see if you can find something related to updating, then you can check the status of the service for logs. You can also try asking Tuxedo support, who may have more insight into this problem.
Aha, that is exactly it! apt-get is called by their driver update tool, Tomte (https://www.tuxedocomputers.com/en/Infos/Help-Support/Frequently-asked-questions/What-is-TUXEDO-Tomte-.tuxedo). I actually would’ve seen this from the
ps auxf | lesscommand above if I had used my eyes a little better, as the top line in the output I pasted above is a sub-process running under Tomte.Checking
systemctl status tuxedo-tomte.serviceyields the following output:aug. 08 15:33:56 laptop systemd[1]: Starting Tomte-daemon, finishes tasks that could not be accomplished before... aug. 08 15:34:06 laptop tuxedo-tomte[1393]: no network found!! some fixes might not be applied correctly aug. 08 15:34:06 laptop tuxedo-tomte[1393]: systemctlCmd: systemd-run --on-active="30sec" tuxedo-tomte configure all >/dev/null 2>&1I guess contacting their support directly is the best way to solve this at this point.
Oh fuck. That’s actually bad. Just to confirm, when you installed the system you did not manually set the size of any of your partitions, correct?
Oh boy, don’t like that sound of that…
That is correct. I used WebFAI to install, the automatic installer provided by Tuxedo Computers (https://www.tuxedocomputers.com/en/TUXEDO-WebFAI.tuxedo).
There are three partition on my primary SSD. One that is mounted to
/boot(1G), one mounted to/boot/efi(512M) and my main partition that is encrypted. I’m guessing it is one of the two boot partitions that don’t have enough space to updateinitramfs-tools?