MariaDB won't start, error 28 no spae for ./ibtmp1

[CDjgould]

Provider of my VPS had a network outage/issue. I don't know what happened exactly but I'm guessing the database got into a weird state trying to restart and having some pending transactions to write. Well it won't write them because it appears it can't set "/ibtmp1" to the necessary file size of 12MB. I don't know where it's trying to create this file or what to do.

root@server:~$ sudo docker logs --tail=20 -f services_global-db_1
2022-02-16 21:15:09 140701531916224 [Note] InnoDB: Number of pools: 1
2022-02-16 21:15:09 140701531916224 [Note] InnoDB: Using SSE2 crc32 instructions
2022-02-16 21:15:09 140701531916224 [Note] InnoDB: Initializing buffer pool, total size = 256M, instances = 1, chunk size = 128M
2022-02-16 21:15:09 140701531916224 [Note] InnoDB: Completed initialization of buffer pool
2022-02-16 21:15:09 140700791240448 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority().
2022-02-16 21:15:09 140701531916224 [Note] InnoDB: Highest supported file format is Barracuda.
2022-02-16 21:15:09 140701531916224 [Note] InnoDB: Starting crash recovery from checkpoint LSN=8430692024
2022-02-16 21:15:10 140701531916224 [Note] InnoDB: Transaction 5505540 was in the XA prepared state.
2022-02-16 21:15:10 140701531916224 [Note] InnoDB: 1 transaction(s) which must be rolled back or cleaned up in total 0 row operations to undo
2022-02-16 21:15:10 140701531916224 [Note] InnoDB: Trx id counter is 5506048
2022-02-16 21:15:10 140701531916224 [Note] InnoDB: Last binlog file '/var/log/mysql/mariadb-bin.000056', position 89702125
2022-02-16 21:15:10 140701531916224 [Note] InnoDB: 128 out of 128 rollback segments are active.
2022-02-16 21:15:10 140700712695552 [Note] InnoDB: Starting in background the rollback of recovered transactions
2022-02-16 21:15:10 140700712695552 [Note] InnoDB: Rollback of non-prepared transactions completed
2022-02-16 21:15:10 140701531916224 [Note] InnoDB: Creating shared tablespace for temporary tables
2022-02-16 21:15:10 140701531916224 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ...
2022-02-16 21:15:10 140701531916224 [ERROR] InnoDB: preallocating 12582912 bytes for file ./ibtmp1 failed with error 28
2022-02-16 21:15:10 140701531916224 [ERROR] InnoDB: Could not set the file size of './ibtmp1'. Probably out of disk space
2022-02-16 21:15:10 140701531916224 [ERROR] InnoDB: Unable to create the shared innodb_temporary
2022-02-16 21:15:10 140701531916224 [ERROR] InnoDB: Plugin initialization aborted with error Generic error
2022-02-16 21:15:15 140701531916224 [Note] InnoDB: Starting shutdown...
2022-02-16 21:15:15 140701531916224 [Note] InnoDB: Removed temporary tablespace data file: "ibtmp1"
2022-02-16 21:15:15 140701531916224 [ERROR] Plugin 'InnoDB' init function returned error.
2022-02-16 21:15:15 140701531916224 [ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed.
2022-02-16 21:15:15 140701531916224 [Note] Plugin 'FEEDBACK' is disabled.
2022-02-16 21:15:15 140701531916224 [ERROR] Unknown/unsupported storage engine: InnoDB
2022-02-16 21:15:15 140701531916224 [ERROR] Aborting

[CDjgould]

figured it out. My VPS is supposed to have 200GB, but I have a disk that is only 10GB. My other server with them shows correctly so put a ticket in with them as I don't think this is something EE did.

root@server:~$ df -h Filesystem Size Used Avail Use% Mounted on /dev/ploop14066p1 9.8G 9.8G 0 100% / none 512M 0 512M 0% /sys/fs/cgroup none 512M 0 512M 0% /dev tmpfs 512M 0 512M 0% /dev/shm tmpfs 512M 1.6M 511M 1% /run tmpfs 5.0M 0 5.0M 0% /run/lock none 512M 0 512M 0% /run/shm
Had to find some data (log files) to delete so I could get the DB to start. Once I deleted a log (/var/log) it had enough space to create the tmp file and start the DB.