0

In the unittest for my DoFo, is it possible to assert the output matches the expected output by comparing the serialized representations of the records?

I've defined a record which uses a default avro coder e.g

@DefaultCoder(AvroCoder.class)
public class JobLogMessage {
  @Nullable
  public String projectId;

  ...
}

I'm writing a unittest for a DoFn that uses DoFnTester to produce a list of JobLogMessage's e.g.

JogLogTransforms.ParsJsonDoFn doFn = new JogLogTransforms.ParsJsonDoFn();
DoFnTester<String, JobLogMessage> fnTester = DoFnTester.of(doFn);
List<JobLogMessage> outputs = fnTester.processBatch(inputs.toArray(new String[inputs.size()]));

I'd like to verify that outputs matches the expected outputs. However if I just use assertEquals I think that will use the equals method which won't correctly evaluate equality unless I explicitly overload equals in JobLogMessage.

What I'd like to do is compare the expected and actual JobLogMessage's by comparing the serialized byte representations produced by AvroCoder. Does Dataflow produce any convenience methods for this?

4

2 に答える 2

1

決定論的スキーマを要求するという制限に問題がない場合は、コードをいくらか単純化して、SDK および JDK で使用可能なユーティリティをより有効に活用できます。

public boolean equals(Object obj) {
  if (!(obj instanceof JobLogMessage)) {
    return false;
  }

  JobLogMessage other = (JobLogMessage) obj;
  AvroCoder<JobLogMessage> coder = AvroCoder.of(JobLogMessage.class);
  return Arrays.equals(CoderUtils.encodeToByteArray(this, coder),
                       CoderUtils.encodeToByteArray(obj, coder));
}

そうは言っても、あなたの目標のために明示的に設計された apache commons-lang のEqualsBuilderやHashCodeBuilderのようなものを使用する方がはるかに良いと思います (コーダーはハッシャーや等価テスターとして使用するように設計されていません) - ただ使用してreturn EqualsBuilder.reflectionEquals(this, obj)ください。

于 2015-03-30T15:31:11.563 に答える
0

AvroCoder を使用して equals を実装し、シリアル化された表現を生成してから、シリアル化された表現を比較しました。

 @Override
  public boolean equals(Object obj) {
    if (!(obj instanceof JobLogMessage)) {
      return false;
    }

    JobLogMessage other = (JobLogMessage) obj;
    AvroCoder<JobLogMessage> coder = AvroCoder.of(JobLogMessage.class);

    Coder.Context context = new Coder.Context(true);
    ByteArrayOutputStream thisStream = new ByteArrayOutputStream();
    try {
      coder.encode(this, thisStream, context);
    } catch (IOException e) {
      throw new RuntimeException("There was a problem encoding the object.", e);
    }

    ByteArrayOutputStream otherStream = new ByteArrayOutputStream();
    try {
      coder.encode(other, otherStream, context);
    } catch (IOException e) {
      throw new RuntimeException("There was a problem encoding the object.", e);
    }

    byte[] theseBytes = thisStream.toByteArray();
    byte[] otherBytes = otherStream.toByteArray();

    if (theseBytes.length != otherBytes.length) {
      return false;
    }
    for (int i = 0; i < theseBytes.length; ++i) {
      if (theseBytes[i] != otherBytes[i]) {
        return false;
      }
    }
    return true;
  }
于 2015-03-29T21:50:05.333 に答える