Spark を使用して Teradata テーブルを読み込んでいます。これが私のコードです:
spark.read.format("jdbc")
.option("url", "jdbc:teradata://127.0.0.1/database=test, TMODE=TERA")
.option("username", "test")
.option("password", "test")
.option("dbtable", "(SELECT TOP 5 * FROM test.users) AS tmp")
.option("driver", "com.teradata.jdbc.TeraDriver")
.load().show()
そして、私はこのエラーを受け取ります:
Caused by: java.sql.SQLException: [Teradata Database] [TeraJDBC 16.20.00.08] [Error 5628] [SQLState HY000] Column Login Date not found in tmp.
at com.teradata.jdbc.jdbc_4.util.ErrorFactory.makeDatabaseSQLException(ErrorFactory.java:309)
at com.teradata.jdbc.jdbc_4.statemachine.ReceiveInitSubState.action(ReceiveInitSubState.java:103)
at com.teradata.jdbc.jdbc_4.statemachine.StatementReceiveState.subStateMachine(StatementReceiveState.java:311)
ただし、Teradata テーブルからわかることは、必要なすべての列があることです。
spark.read.format("jdbc")
.option("url", "jdbc:teradata://127.0.0.1/database=test, TMODE=TERA")
.option("username", "test")
.option("password", "test")
.option("dbtable", "(SELECT * FROM dbc.columnsV WHERE databasename = 'test' AND tablename = 'users') tmp")
.option("driver", "com.teradata.jdbc.TeraDriver")
.load().show()
出力:
+-------------------+-------------------------+----------------------+------------+---------------------------------+---------------+----------+-------------+------------+------------+--------+-------------+------------------+...
|DatabaseName |TableName |ColumnName |ColumnFormat|ColumnTitle |SPParameterType|ColumnType|ColumnUDTName|ColumnLength|DefaultValue|Nullable|CommentString|DecimalTotalDigits|...
+-------------------+-------------------------+----------------------+------------+---------------------------------+---------------+----------+-------------+------------+------------+--------+-------------+------------------+...
|test |users |active_flag |-(18)9 |Is User Active | |D |null |8 |null |Y |null |18 |...
|test |users |user_id |-(18)9 |User Identifier | |D |null |8 |null |Y |null |18 |....
|test |users |login_date |yyyy-mm-dd |Login Date | |DA |null |4 |null |Y |null |null |....
+-------------------+-------------------------+----------------------+------------+---------------------------------+---------------+----------+-------------+------------+------------+--------+-------------+------------------+....
Spark がテーブルを読み取れない理由と列を取得できない理由はありますか?