dedup
Use the dedup command to remove results that contain
identical combination of field values based on the search order generated through the
sort command.
Syntax
dedup <dedup_options> <field_name> [, <field_name>, ...]Parameters
The following table lists the parameters used in this command, along with their descriptions.
| Parameter | Description |
|---|---|
|
|
Specify the field whose values must be checked for duplicates. |
|
|
Syntax: [count =
<count>][includenulls =
[true|false]] [consecutive =
[true|false]]
|
The following query groups logs by each unique combination of client host city and IP, calculates the sum of content size for each group, sorts each group by descending order of content size, and finally removes duplicate rows for a client host city. This effectively retains only those rows that correspond to the highest content size for each client host city:
* | stats sum('Content Size') as 'Content Size' by 'Client Host City', 'Source IP'
| sort -'Content Size'
| dedup 'Client Host City'With the above query, the resulting records table has three columns Client
Host City, Source IP, and Content
Size.
If you specify the dedup option count = 2, then 2
rows that have the same value of Client Host City are
available.
If you specify the dedup option includenulls =
true, then those rows are included where Client Host City
value is null.
If you specify the dedup option consecutive = true,
then only those rows are removed where the consecutive values of Client Host
City are the same.