2

I am using jpa with hibernate using seam. Ineed to insert 200,000 records at a time.

Here is my code:

hibernate.cfg.xml

<hibernate-configuration>
 <session-factory name="java:/mobeeSessionFactory">
  <property name="hibernate.connection.pool_size">10</property>
  <property name="hibernate.connection.driver_class">oracle.jdbc.OracleDriver</property>
  <property name="hibernate.connection.url">jdbc:oracle:thin:@localhost:1521:mobee</property>
  <property name="hibernate.connection.username">mobeemigrate</property>
  <property name="hibernate.connection.password">mobeemigrate</property>
  <property name="hibernate.show_sql">false</property>
  <property name="hibernate.dialect">org.hibernate.dialect.OracleDialect</property>
  <property name="hibernate.format_sql">false</property>
  <property name="hibernate.hbm2ddl.auto">update</property>
  <property name="hibernate.session_factory_name">java:/mobeeSessionFactory</property>
  <property name="hibernate.connection.datasource">mobeeadminDataSource</property>
  <property name="hibernate.jdbc.batch_size">10000</property>
  <property name="hibernate.cache.use_first_level_cache">true</property>
  <property name="hibernate.cache.use_second_level_cache">false</property>
  <property name="hibernate.transaction.manager_lookup_class">org.hibernate.transaction.JBossTransactionManagerLookup</property>
  <property name="hibernate.transaction.factory_class">org.hibernate.transaction.JTATransactionFactory</property>
  <property name="hibernate.transaction.auto_close_session">false</property>
  <property name="hibernate.cache.provider_class">org.hibernate.cache.HashtableCacheProvider</property>
  <property name="hibernate.transaction.flush_before_completion">true</property>

  <!-- Here are the mappings -->
  <mapping package="com.manam.mobee.persist.entity"/>
  <mapping class="com.manam.mobee.persist.entity.TempCustomers"/>
  <mapping class="com.manam.mobee.persist.entity.TempAccounts"/>
 </session-factory>

component.xml

<persistence:hibernate-session-factory name="hibernateSessionFactory" cfg-resource-name="hibernate.cfg.xml"/>
    <persistence:managed-hibernate-session name="session" 
                 auto-create="true"
     session-factory-jndi-name="java:/mobeeSessionFactory"/>

Sample Code:

 Session session =hibernateSessionFactory.openSession();
             Transaction tx = session.beginTransaction();

   for(int i=0;i<doTempAccounts.size();i++){

     try {
    TempAccounts temp=new TempAccounts();

    BeanUtils.copyProperties(temp, doTempAccounts.get(i));

    session.save(temp);

    if ( i % 10000 == 0 ) { //10000, same as the JDBC batch size
     //flush a batch of inserts and release memory:
    log.info("********** Batch Updates**********");
    session.flush();
    session.clear();
     }
     }
     }
    tx.commit();
    session.close();

The above code is working but it takes around 10 minutes to insert 200,000 records. Is there any configuration in my code to increase the performance of bulk inserts?

In above code I do a session.flush() every 10000 records but its not insert Database every time. Can you explain how to perform bulk inserts to db.?

4

3 に答える 3

1

In above code I do a session.flush() every 10000 records but its not insert Database every time.

I presume that you mean that the records don't appear to be there until you reach the end; i.e. they don't show up in a query made in a different transaction.

If that is what you mean, then the explanation is simple. You are doing the insert in a single transaction, and the records only become visible when the transaction is committed ... right at the end. Try splitting the inserts into multiple transactions if you need them to be visible sooner.

You should also read the Hibernate documentation on batch operations - http://docs.jboss.org/hibernate/orm/3.3/reference/en/html/batch.html. I note that it says that a reasonable batch size is 20 to 50 ... not 10000. It also mentions the approaches of using SQL directly for batch data operations and StatelessSession.

Here's another resource which is specific to insertion and includes insertion with StatelessSession: http://javainnovations.blogspot.com.au/2008/07/batch-insertion-in-hibernate.html

于 2012-12-08T00:24:47.777 に答える
1

Look into StatelessSession and how to use it to insert items into your DB. This appears to be the go to answer when this question is asked and I personally just used it to make a 15 minute job complete in about 15 seconds. It doesn't fit every scenario, but if it does, it works great.

于 2012-12-08T00:49:46.010 に答える
0

You can do it almost the same way you do in SQL. Look at the following HQL example:

insert into DelinquentAccount (id, name) select c.id, c.name from Customer c where ...

Reference: http://docs.jboss.org/hibernate/orm/4.0/devguide/en-US/html/ch04.html#d0e2184

于 2014-07-18T13:24:44.420 に答える