5

After processing data with: input | filter | output > ElasticSearch the format it's get stored in is somewhat like:

"_index": "logstash-2012.07.02",
"_type": "stdin",
"_id": "JdRaI5R6RT2do_WhCYM-qg",
"_score": 0.30685282,
"_source": {
    "@source": "stdin://dist/",
    "@type": "stdin",
    "@tags": [
        "tag1",
        "tag2"
    ],
    "@fields": {},
    "@timestamp": "2012-07-02T06:17:48.533000Z",
    "@source_host": "dist",
    "@source_path": "/",
    "@message": "test"
}

I filter/store most of the important information in specific fields, is it possible to leave out the default fields like: @source_path and @source_host? In the near future it's going to store 8 billion logs/month and I would like to run some performance tests with this default fields excluded (I just don't use these fields).

4

3 に答える 3

7

これにより、出力からフィールドが削除されます。

filter {
    mutate {
        # remove duplicate fields
        # this leaves timestamp from message and source_path for source
        remove => ["@timestamp", "@source"]
    }
 }
于 2013-03-25T21:41:13.070 に答える
0

その一部は、ログを表示するために使用している Web インターフェイスによって異なります。私は Kibana と、次のインデックスを作成する顧客ロガー (c#) を使用しています。

{
  "_index": "logstash-2013.03.13",
  "_type": "logs",
  "_id": "n3GzIC68R1mcdj6Wte6jWw",
  "_version": 1,
  "_score": 1,
  "_source": 
  {
    "@source": "File",
    "@message": "Shalom",
    "@fields": 
    {
      "tempor": "hit"
    },
    "@tags": 
    [
      "tag1"
    ],
    "level": "Info"
    "@timestamp": "2013-03-13T21:47:51.9838974Z"
  }
}

これは Kibana に表示され、ソース フィールドはありません。

于 2013-03-13T22:30:15.030 に答える