1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
|
.\" Copyright Vespa.ai. Licensed under the terms of the Apache 2.0 license. See LICENSE in the project root.
.\" $Id: logfmt.1,v 1.12 2007-06-19 09:37:25 daljord Exp $
.\"
.Dd Jun 7, 2017
.Dt vespa-logfmt \&1 "Vespa documentation"
.Os "Yahoo! Vespa" "6.0"
.Os
.Sh NAME
.Nm vespa-logfmt
.Nd select and format messages from Vespa log files
.Sh SYNOPSIS
.Nm
.Op Fl l Ar levellist
.Op Fl s Ar fieldlist
.Op Fl p Ar pid
.Op Fl S Ar service
.Op Fl H Ar host
.Op Fl c Ar regex
.Op Fl m Ar regex
.Op Fl t
.Op Fl f
.Op Fl L
.Op Fl N
.Op Fl ts
.Op Ar
.Sh DESCRIPTION
The
.Nm
utility reads Vespa log files, select messages and writes a formatted
version of selected messages to the standard output.
.Pp
The options are as follows:
.Bl -tag -width ".It Fl l Ar levellist"
.It Fl l Ar levellist
Select which log levels to select.
The default is to select "fatal", "error", "warning" and "info"
levels, and suppress "config", "event", "debug" and "spam" levels;
but when using this option, only the named levels will be selected.
The
.Ar levellist
is a comma separated list of level names.
The name
.Em all
may be used to add all known levels.
Prepending a minus sign will deselect the level named.
Starting the list with a plus sign will add and remove levels
from the current (or default) list of levels instead
of replacing it.
.It Fl s Ar fieldlist
Select which fields of log messages to show.
The order of the actual output fields is fixed.
When using this option, only the named fields will be shown. The
fieldlist is a comma separated list of field names. The name
.Em all
may be used to add all possible fields.
Prepending a minus sign will turn off display of the named field.
Starting the list with a plus sign will add and remove fields
from the current (or default) list of fields instead
of replacing it.
.Pp
The fields which may be named are:
.Bl -tag -width component
.It time
Print the time in seconds since the epoch.
Ignored if
.Em fmttime
is shown.
.It fmttime
Print the time in human-readable [YYYY-MM-DD HH:mm:ss] format.
Note that the time is printed in the local timezone; to get GMT
output use
.Nm "\*[q]env TZ=GMT vespa-logfmt\*[q]"
as your command.
.It msecs
Add milliseconds after the seconds in
.Em time
and
.Em fmttime
output. Ignored if
.Em usecs
is in effect.
.It usecs
Add microseconds after the seconds in
.Em time
and
.Em fmttime
output.
.It host
Print the hostname field.
.It level
Print the level field (uppercased).
.It pid
Print the pid field.
.It service
Print the service field.
.It component
Print the component field.
.It message
Print the message text field.
You probably always want to add this.
.El
.Pp
Using this option several times works as if the given
.Ar fieldlist
arguments had been concatenated into one comma-separated list.
The default fields to show are as if
.Bk
.Op Fl s Ar fmttime,msecs,level,service,component,message
.Ek
had been given.
.It Fl p Ar pid
Select only messages where the pid field matches the
.Ar pid
string exactly.
.It Fl S Ar service
Select only messages where the service field matches the
.Ar service
string exactly.
.It Fl H Ar host
Select only messages where the hostname field matches the
.Ar host
string exactly.
.It Fl c Ar regex
Select only messages where the component field matches the
.Ar regex
given, using
.Xr perlre
regular expression matching.
.It Fl m Ar regex
Select only messages where the message text field matches the
.Ar regex
given, using
.Xr perlre
regular expression matching.
.It Fl f
Invoke tail -F to follow input file
.Bk
.Fl "s +host"
.Ek
as default.
.It Fl N
Dequote quoted newlines in the message text field to an actual newline plus tab.
.It Fl t
Format the component field (if shown) as a fixed-with string,
truncating if necessary.
.It Fl ts
Format the service field (if shown) as a fixed-with string,
truncating if necessary.
.El
.Sh EXAMPLES
The command:
.Pp
.Bd -literal -offset indent
vespa-logfmt -l event -s service,message,fmttime,message
.Ed
.Pp
will display only messages with log level "event",
printing a human-readable time (without any fractional seconds),
the service generating the event and the event message, like this:
.Bd -literal -offset indent
[2004-12-07 18:43:01] config-sentinel starting/1 name="logd"
[2004-12-07 18:43:01] logd started/1 name="logdemon"
[2004-12-07 18:45:51] rtc starting/1 name="rtc.index0"
[2004-12-07 18:45:51] rtc.index0 started/1 name="flexindexer.index"
[2004-12-07 18:45:51] rtc.index0 stopping/1 name="flexindexer.index" why="done"
[2004-12-07 18:45:53] rtc stopped/1 name="rtc.index0" pid=50600 exitcode=0
[2004-12-07 18:46:13] logd stopping/1 name="logdemon" why="done ok."
[2004-12-07 18:46:13] config-sentinel stopped/1 name="logd" pid=49633 exitcode=0
.Ed
.Pp
Note that the second "message" item in the fieldlist is redundant,
and that order of printed field is fixed no matter what the fieldlist
order is.
.Pp
The command:
.Pp
.Bd -literal -offset indent
vespa-logfmt -l all-info,-debug -s level \e
-s time,usecs,component,message -t -l -event
.Ed
.Pp
will display messages with log levels that are
.Em not
any of
.Em info, debug,
or
.Em event,
printing the time in seconds and microseconds, the log level, the
component name, and the message text, possibly somewhat like this:
.Bd -literal -offset indent
1102441382.530423 CONFIG : nc Config handle: 'pandora.0-rtx'
1102441551.471568 CONFIG : flexindexer.doc Adding document type typetest-0
1102441573.148211 WARNING : logdemon stopping on signal 15
1102441887.158000 WARNING : com.yahoo.fs4.m read exception
1102441935.569567 WARNING : rtc Dispatch inherited job failed for dir dispatch0
1102442474.205920 WARNING : rtx RTC (tcp/172.24.94.75:10161) : DOWN
1102442983.075669 ERROR : flexindexer.std Unable to find cluster map defaultcluster
.Ed
.Sh FILES
If no file argument is given,
.Nm
will read the last vespa log file $VESPA_HOME/logs/vespa/vespa.log (this also works with the
.Fl f
option).
Otherwise, reads only the files given as arguments.
To read standard input, supply a single dash '-' as a file argument.
.Sh SEE ALSO
Documentation in the "log" module for input file format.
.Sh HISTORY
Developed as part of Vespa 1.1. The default output format
reflects the old "fastlib" log formatting, with minor differences
and is intended to be human-readable, not parsed.
|
