[Question] Query whole log file using serverquery

How can I query a whole log file using the logview command?
I know, i have to use the command multiple times, but I was not quite able to figure out how to query the whole file without it leaving some stuff out at the beginning of the file

Please take a look at the explanaintion by ScP.

Logview command usage

serveradmin> help logview
Usage: logview [lines={1-100}] [reverse={1|0}] [instance={1|0}] [begin_pos={n}]

Permissions:
  b_serverinstance_log_view
  b_virtualserver_log_view

Description:
  Displays a specified number of entries from the servers log. If instance is
  set to 1, the server will return lines from the master logfile
  (ts3server_0.log) instead of the selected virtual server logfile.

Parameters:
  lines : integer : reduce the number of lines in response; Default `100`
  reverse : integer : when `1`, reverse the order of lines in response
  instance : integer : when `1`, return log from instance; otherwise from virtual server
  begin_pos : integer : skip `n` number of lines

Example:
  logview lines=30
  last_pos=403788 file_size=411980 l=\p\slistening\son\s0.0.0.0:9987 ...
  error id=0 msg=ok

As ScP mentioned already the wrong documentation, begin_pos is the position of the cursor inside the logfile while catching (reading) it on your request not the lines to skip as I assumed.

How to build the request?

With that knowledge you can now easily build the request structure.
For example:

serveradmin > logview lines=1
last_pos=1771 file_size=1931 l=2021-02-04\s11:33:08.792248\pINFO\s\s\s\s\p\s\s\s\s\s\s\s\s\s\s\s\s\s\s\p\s\s\s\pmyTeamSpeak\sidentifier\srevocation\slist\swas\sdownloaded\ssuccessfully\s-\sall\srelated\sfeatures\sare\sactivated

The response includes last_pos which is the exact position BEFORE the line you can see as response.
Use this information in the query for the next request.

serveradmin > logview lines=1 begin_pos=1771
last_pos=1626 file_size=1931 l=2021-02-03\s20:28:55.077171\pINFO\s\s\s\s\pCIDRManager\s\s\s\p\s\s\s\pupdated\squery_ip_allowlist\sips:\s127.0.0.1\/32,\s::1\/128,\s*.*.*.*\/32,\*.*.*.*\/32,\s

The response for this request is the next line as you can see in my example.
last_pos defines the exact position BEFORE the line you can see again.
Requesting the next line is similar to what I’ve explained above.

serveradmin > logview lines=1 begin_pos=1626
last_pos=1514 file_size=1931 l=2021-02-03\s20:28:55.077171\pINFO\s\s\s\s\pQuery\s\s\s\s\s\s\s\s\s\p\s\s\s\plistening\sfor\shttp\squery\son\s*.*.*.*:****,\s[::]:****

last pos

As you can see the value for last_pos is getting smaller in my examples.
The reason for this is by default the file is being read in reverse order - starting at the end of the file.
The cursor is moving right into the direction to the beginning of the file.
The value itself describes the filesize (in bytes) until that currently selected position.


Conclusion:

As written in my example you will reveive the lines of the logfile in reverse order.
Starting by the last line moving into the direction to the first line.

Looping through the file

To automate the process to receive the whole logfile you can simply create a loop.
Just send the command logview inside your loop adjusting it’s parameters based on the response.

logview lines=100 begin_pos=0

The response will include last_pos (e.g. last_pos=1234).
The next iteration of your loop will set begin_pos=last_pos (e.g. begin_pos=1234).
As long as begin_pos is not equal to 0 you have not reached the end of the file.
The last line will return last_pos with a value of 0. That’s the end of the file, you can break the loop now.

That’s it.

Thank you for your very detailed response. I get that I was on the right track with my implementation as I was already using the begin_pos = last_pos pattern, but the key here seems to be to only get one line at a time after the first query, since I was getting incomplete lines when querieing multiple lines after the first query

Thanks so much. Will implement

MfG

1 Like

I implemented it as you described it. But the first line in the logfile is still missing the first three characters. And the last last_pos is 0 so I should have reached to end but where are my three bytes? I really want them since they are part of the timestamp and I can’t parse it without those three characters

Is it possible the logfile is corrupted or let’s say formatted wrongly?

The pre-last entry should contain last_pos for the last line.
If not try to catch more than 1 line.
Catching 1 line only isn’t effient anyway and was just an example.


(edit)

Well I’ve noticed the last line is being cut off as your described.
No matter you’re catching 1 line only or 100 lines.
The first 3 characters are missing.
Reading the instance log results to the same issue, no matter how big the logfile is.

Looks like a bug to me.

I wonder if @TS.ChrisR can reproduce this.

1 Like

Looks good for me

Log 1 issued with 99 lines

2021-02-05\s07:53:18.366145\pINFO\s\s\s\s\pVirtualServer\s\p1\s\s\pclient\s'Chris'(id:3)\swas\sremoved\sfrom\sservergroup\s'Normal'(id:11)\sby\sclient\s'Chris'(id:3)
2021-02-05\s07:53:25.423506\pINFO\s\s\s\s\pQuery\s\s\s\s\s\s\s\s\s\p1\s\s\pquery\sfrom\s\s1\s[::1]:6026\sissued:\slogview\slines=99



2021-02-05 07:53:18.366145|INFO    |VirtualServer |1  |client 'Chris'(id:3) was removed from servergroup 'Normal'(id:11) by client 'Chris'(id:3)
2021-02-05 07:53:25.423506|INFO    |Query         |1  |query from  1 [::1]:6026 issued: logview lines=99

Log 0 issued with 99 lines

2021-02-05 07:51:05.114951|INFO    |Query         |   |query from 1 [::1]:6026 issued: login with account "serveradmin"(serveradmin)
2021-02-05 07:51:10.377080|INFO    |Query         |   |query from  1 [::1]:6026 issued: use 1



2021-02-05\s07:51:05.114951\pINFO\s\s\s\s\pQuery\s\s\s\s\s\s\s\s\s\p\s\s\s\pquery\sfrom\s1\s[::1]:6026\sissued:\slogin\swith\saccount\s"serveradmin"(serveradmin)
2021-02-05\s07:51:10.377080\pINFO\s\s\s\s\pQuery\s\s\s\s\s\s\s\s\s\p\s\s\s\pquery\sfrom\s\s1\s[::1]:6026\sissued:\suse\s1
1 Like

Thanks for your reply. That’s indeed strange…

serveradmin> use 0
error id=0 msg=ok
serveradmin> logview
last_pos=0 file_size=2090 l=1-02-03\s20:28:54.030538\pINFO\s\s\s\s\pServerLibPriv\s\p\s\s\s\pTeamSpeak\s3\sServer\s3.13.3\s(2020-12-16\s14:17:05)|l=2021-02-03\s20:28:54.030538\pINFO\s\s\s\s\pServerLibPriv\s\p\s\s\s\pSystemInformation:\sWindows\sServer\s2016\s(17763)\sx64\s(AMD\sor\sIntel)\sBinary:\s64bit|l=2021-02-03\s20:28:54.030538\pINFO\s\s\s\s\pServerLibPriv\s\p\s\s\s\pUsing\shardware\saes|l=2021-02-03\s20:28:54.093021\pINFO\s\s\s\s\pDatabaseQuery\s\p\s\s\s\pdbPlugin\sname:\s\s\s\sMariaDB\splugin,\sversion\s3,\s(c)TeamSpeak\sSystems\sGmbH|l=2021-02-03\s20:28:54.093021\pINFO\s\s\s\s\pDatabaseQuery\s\p\s\s\s\pdbPlugin\sversion:\s2|l=2021-02-03\s20:28:54.296093\pINFO\s\s\s\s\pAccounting\s\s\s\s\p\s\s\s\pLicensing\sInformation|l=2021-02-03\s20:28:54.296093\pINFO\s\s\s\s\pAccounting\s\s\s\s\p\s\s\s\ptype\s\s\s\s\s\s\s\s\s\s\s\s\s\s:\sNon-Profit\sLicense|l=2021-02-03\s20:28:54.296093\pINFO\s\s\s\s\pAccounting\s\s\s\s\p\s\s\s\pstarting\sdate\s\s\s\s\s:\sTue\sDec\s\s1\s00:00:00\s2020|l=2021-02-03\s20:28:54.296093\pINFO\s\s\s\s\pAccounting\s\s\s\s\p\s\s\s\pending\sdate\s\s\s\s\s\s\s:\sMon\sJun\s21\s00:00:00\s2021|l=2021-02-03\s20:28:54.296093\pINFO\s\s\s\s\pAccounting\s\s\s\s\p\s\s\s\pmax\svirtualservers:\s10|l=2021-02-03\s20:28:54.296093\pINFO\s\s\s\s\pAccounting\s\s\s\s\p\s\s\s\pmax\sslots\s\s\s\s\s\s\s\s\s:\s512|l=2021-02-03\s20:28:54.905309\pINFO\s\s\s\s\p\s\s\s\s\s\s\s\s\s\s\s\s\s\s\p\s\s\s\pPuzzle\sprecompute\stime:\s519|l=2021-02-03\s20:28:54.905309\pINFO\s\s\s\s\pFileManager\s\s\s\p\s\s\s\plistening\son\s*.*.*.*:****|l=2021-02-03\s20:28:54.905309\pINFO\s\s\s\s\pQuery\s\s\s\s\s\s\s\s\s\p\s\s\s\pUsing\sa\squery\sthread\spool\ssize\sof\s2|l=2021-02-03\s20:28:55.077171\pINFO\s\s\s\s\pQuery\s\s\s\s\s\s\s\s\s\p\s\s\s\plistening\sfor\squery\son\s*.*.*.*:****,\s[::]:****|l=2021-02-03\s20:28:55.077171\pINFO\s\s\s\s\pQuery\s\s\s\s\s\s\s\s\s\p\s\s\s\plistening\sfor\sssh\squery\son\s*.*.*.*:****,\s[::]:****|l=2021-02-03\s20:28:55.077171\pINFO\s\s\s\s\pQuery\s\s\s\s\s\s\s\s\s\p\s\s\s\plistening\sfor\shttp\squery\son\s*.*.*.*:****,\s[::]:****|l=2021-02-03\s20:28:55.077171\pINFO\s\s\s\s\pCIDRManager\s\s\s\p\s\s\s\pupdated\squery_ip_allowlist\sips:\s127.0.0.1\/32,\s::1\/128,\s*.*.*.*\/32,\s*.*.*.*\/32,\s|l=2021-02-04\s11:33:08.792248\pINFO\s\s\s\s\p\s\s\s\s\s\s\s\s\s\s\s\s\s\s\p\s\s\s\pmyTeamSpeak\sidentifier\srevocation\slist\swas\sdownloaded\ssuccessfully\s-\sall\srelated\sfeatures\sare\sactivated|l=2021-02-05\s11:33:08.188470\pINFO\s\s\s\s\p\s\s\s\s\s\s\s\s\s\s\s\s\s\s\p\s\s\s\pmyTeamSpeak\sidentifier\srevocation\slist\swas\sdownloaded\ssuccessfully\s-\sall\srelated\sfeatures\sare\sactivated
error id=0 msg=ok

As you can see the first 3 characters of the last possible line are missing.

Probably a Windows related issue only?

Not tested this on a Linux instance so far…
There’s no formatting issue in the file.
I’ve deleted the logfile and restarted the server to test this out,

(start parameter: logappend=1)


Not that this issue is important but… well it’s there.

I did it on linux so this is not os dependend

twitch instagram twitter facebook