Bug #1243

Server crash related to filesystem access

Added by Spyhawk 21 days ago. Updated 5 days ago.

Status:New% Done:

0%

Priority:Urgent
Assignee:-
Category:Server
Target version:2.77
OS: Arch:

Description

The test server tends to crash every few weeks or so. The logs have interesting content:

  • it seems the game log file can’t be opened (maybe some issue with log rotation)
  • the mapvote.txt suddenly can’t be opened
  • the database eventually can’t be accessed anymore (locked), resulting in a hard crash
  • There are some WA error, but I believe this is unreletad (WA got some bugfix in its dev branch)

See attachment, last bit of the ~80mb server log file.

output.log.txt Magnifier (751 KB) Spyhawk, 02.04.2019 18:23


Related issues

Related to ET: Legacy Development - Bug #987: Server crash Feedback 01.02.2017

Associated revisions

Revision c6d650ea
Added by Spyhawk 11 days ago

game: fixed mod logfile not always created, refs #1243

History

#1 Updated by Spyhawk 21 days ago

For reference, there is a copy of the two previous crashed logs on the server (internal access, 80+ mb each):
  • output_crashed_at_0820_on_270119.log
  • output_crashed_at_1112_on_010419.log

Additional info:
Restarted the test server.
`WARNING: Couldn’t open logfile: legacy_mod.log` is printed, but doesn’t seem to be critical here. (done)

#2 Updated by Spyhawk 11 days ago

About the mod log side-issue: it looks like the "append" mode fails to create a new file when it isn’t already present. For some reason, the "create" fallback just fails.
  • The issue doesn’t happen on windows, but only Linux (confirmed on the test server running CentOS 7, locally on Arch, and on by a third party on his server).
  • this isn’t a permission issue, since it is possible to create a new file (tested locally)

I couldn’t find much about that issue, apart from this article ("The descriptions for both the "a" and "a+" append modes say that the file will be created if it doesn’t exist. I’ve found, in practice, that this doesn’t work. If the file doesn’t exist when fopen is called with a "a" or "a+" open mode, the file will not be created.")

Suggestion for a dirty workaround: if the append mod fails, try creating the file with a normal "write" mode, close the file, and try to append it to it again. (done)

#3 Updated by Spyhawk 5 days ago

Server crashed again. Same symtoms. Log is 95 Mb this time (output_crashed_at_0453_on_190419.log).

A recent commit fixed some issue in the Lua stack (" Lua vm stack was uncontrollably growing after each hook
callback call"), see if that fix improves the situation on the next update.

#4 Updated by Spyhawk 5 days ago

  • Related to Bug #987: Server crash added

Also available in: Atom PDF