apache-kafka - オプションのレコードを含む AVRO スキーマ

Question

こんにちは、次の例の AVRO スキーマを作成する必要があります。

{ "Car" : { "Make" : "Ford" , "Year": 1990 , "Engine" : "V8" , "VIN" : "123123123" , "Plate" : "XXTT9O", 
"Accident" : { "Date" :"2020/02/02" , "Location" : "NJ" , "Driver" : "Joe" } ,
"Owner" :  { "Name" : "Joe" , "LastName" : "Doe" } } 

Accident と Owner はオプション オブジェクトであり、作成されたスキーマも次のサブセット メッセージを検証する必要があります。

{ "Car" : { "Make" : "Tesla" , "Year": 2020 , "Engine" : "4ELEC" , "VIN" : "54545426" , "Plate" : "TESLA" }

私は AVRO の仕様を読み、多くのオプションの属性と配列の例を見ましたが、それらのどれも記録のために機能しませんでした。レコードをオプションとして定義するにはどうすればよいですか? ありがとう。

オプション属性のない次のスキーマは機能しています。

{
  "name": "MyClass",  "type": "record",  "namespace": "com.acme.avro",  "fields": [
    {
      "name": "Car",   "type": {
        "name": "Car","type": "record","fields": [
          { "name": "Make",    "type": "string"   },
          { "name": "Year",    "type": "int"      },
          { "name": "Engine",    "type": "string" },
          { "name": "VIN",    "type": "string"    },
          { "name": "Plate",    "type": "string"  },
          { "name": "Accident",
                "type":
                { "name": "Accident",
                  "type": "record",
                  "fields": [
                       { "name": "Date","type": "string" },
                       { "name": "Location","type": "string" },
                       { "name": "Driver", "type": "string" }
                    ]
                 }
          },

          { "name": "Owner",
            "type":
                {"name": "Owner",
                 "type": "record",
                 "fields": [
                {"name": "Name",  "type": "string" },
                {"name": "LastName", "type": "string" }
              ]
            }
          }
        ]
      }
    }
  ]
}

提案されたように所有者オブジェクトを変更すると、avro-tool がエラーを返します。

      { "name": "Owner",
        "type": [
            "null",
            "record" : {
              "name": "Owner",
              "fields": [
                 {"name": "Name",  "type": "string" },
                 {"name": "LastName", "type": "string" }
               ]
             }
          ] , "default": null  }
       ]
   }
}

] }

テスト：

Projects/avro_test$ java -jar avro-tools-1.8.2.jar fromjson --schema-file CarStackOver.avsc Car.json > o2
log4j:WARN No appenders could be found for logger (org.apache.hadoop.metrics2.lib.MutableMetricsFactory).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
Exception in thread "main" org.apache.avro.SchemaParseException: org.codehaus.jackson.JsonParseException: Unexpected character (':' (code 58)): was expecting comma to separate ARRAY entries
 at [Source: org.apache.hadoop.fs.ChecksumFileSystem$FSDataBoundedInputStream@4034c28c; line: 26, column: 13]
        at org.apache.avro.Schema$Parser.parse(Schema.java:1034)
        at org.apache.avro.Schema$Parser.parse(Schema.java:1004)
        at org.apache.avro.tool.Util.parseSchemaFromFS(Util.java:165)
        at org.apache.avro.tool.DataFileWriteTool.run(DataFileWriteTool.java:83)
        at org.apache.avro.tool.Main.run(Main.java:87)
        at org.apache.avro.tool.Main.main(Main.java:76)
Caused by: org.codehaus.jackson.JsonParseException: Unexpected character (':' (code 58)): was expecting comma to separate ARRAY entries
 at [Source: org.apache.hadoop.fs.ChecksumFileSystem$FSDataBoundedInputStream@4034c28c; line: 26, column: 13]
        at org.codehaus.jackson.JsonParser._constructError(JsonParser.java:1433)
        at org.codehaus.jackson.impl.JsonParserMinimalBase._reportError(JsonParserMinimalBase.java:521)
        at org.codehaus.jackson.impl.JsonParserMinimalBase._reportUnexpectedChar(JsonParserMinimalBase.java:442)
        at org.codehaus.jackson.impl.Utf8StreamParser.nextToken(Utf8StreamParser.java:482)
        at org.codehaus.jackson.map.deser.std.BaseNodeDeserializer.deserializeArray(JsonNodeDeserializer.java:222)
        at org.codehaus.jackson.map.deser.std.BaseNodeDeserializer.deserializeObject(JsonNodeDeserializer.java:200)
        at org.codehaus.jackson.map.deser.std.BaseNodeDeserializer.deserializeArray(JsonNodeDeserializer.java:224)
        at org.codehaus.jackson.map.deser.std.BaseNodeDeserializer.deserializeObject(JsonNodeDeserializer.java:200)
        at org.codehaus.jackson.map.deser.std.BaseNodeDeserializer.deserializeObject(JsonNodeDeserializer.java:197)
        at org.codehaus.jackson.map.deser.std.BaseNodeDeserializer.deserializeArray(JsonNodeDeserializer.java:224)
        at org.codehaus.jackson.map.deser.std.BaseNodeDeserializer.deserializeObject(JsonNodeDeserializer.java:200)
        at org.codehaus.jackson.map.deser.std.JsonNodeDeserializer.deserialize(JsonNodeDeserializer.java:58)
        at org.codehaus.jackson.map.deser.std.JsonNodeDeserializer.deserialize(JsonNodeDeserializer.java:15)
        at org.codehaus.jackson.map.ObjectMapper._readValue(ObjectMapper.java:2704)
        at org.codehaus.jackson.map.ObjectMapper.readTree(ObjectMapper.java:1344)
        at org.apache.avro.Schema$Parser.parse(Schema.java:1032)