JtaEntityManager GC

[cocorossello]

Hi,

We are having some memory problems in our applications. We are using tomee
7.0.4 with eclipselink, no second cache level. Most of the application are
@ApplicationScoped.

Transaction is managed by container (JTA), no extended transactions at all,
everything we do with persistence is through the entity manager, which is
always injected with:
@PersistenceContext(unitName = "travelcPU")
private EntityManager entityManager;


I have isolated one production server, and waited until all sessions are
dead and then flushed all caches and then I took a heapdump. It's still
1.6Gb, even with no information, I think it's quite big.

I can see in the heapdump some 22K
org.apache.openejb.persistence.JtaEntityManager . And then (And I'm not sure
if it's related to JtaEntityManager) a lot of QueryBasedValueHolder which
are in fact holding references to a lot of our domain objects.

So, my question is, shouldn't those JtaEntityManager be cleared on GC? I
think that they don't hold references to the eclipselink EnityManager, so
the problem probably comes from another part, but can you confirm?

Thanks in advance.



--
Sent from: http://tomee-openejb.979440.n4.nabble.com/TomEE-Users-f979441.html

[Romain Manni-Bucau]

Hi,

normally they are garbaged collected once the transaction ends. If you use
it without transaction you can need to call clear() once you are done.


Romain Manni-Bucau
@rmannibucau <https://twitter.com/rmannibucau> | Blog
<https://rmannibucau.metawerx.net/> | Old Blog
<http://rmannibucau.wordpress.com> | Github <https://github.com/rmannibucau> |
LinkedIn <https://www.linkedin.com/in/rmannibucau> | Book
<https://www.packtpub.com/application-development/java-ee-8-high-performance>

Post by cocorossello
Hi,
We are having some memory problems in our applications. We are using tomee
7.0.4 with eclipselink, no second cache level. Most of the application are
@ApplicationScoped.
Transaction is managed by container (JTA), no extended transactions at all,
everything we do with persistence is through the entity manager, which is
@PersistenceContext(unitName = "travelcPU")
private EntityManager entityManager;
I have isolated one production server, and waited until all sessions are
dead and then flushed all caches and then I took a heapdump. It's still
1.6Gb, even with no information, I think it's quite big.
I can see in the heapdump some 22K
org.apache.openejb.persistence.JtaEntityManager . And then (And I'm not sure
if it's related to JtaEntityManager) a lot of QueryBasedValueHolder which
are in fact holding references to a lot of our domain objects.
So, my question is, shouldn't those JtaEntityManager be cleared on GC? I
think that they don't hold references to the eclipselink EnityManager, so
the problem probably comes from another part, but can you confirm?
Thanks in advance.
--
Sent from: http://tomee-openejb.979440.n4.nabble.com/TomEE-Users-
f979441.html

[sgjava]

Have you tried MAT https://www.eclipse.org/mat to analyze the heap?



--
Sent from: http://tomee-openejb.979440.n4.nabble.com/TomEE-Users-f979441.html

[Paul Carter-Brown]

What is your xmx setting and did you force a gc prior to taking the heap
dump?

On your db have you checked if there are any long running transactions

Post by cocorossello
Hi,
We are having some memory problems in our applications. We are using tomee
7.0.4 with eclipselink, no second cache level. Most of the application are
@ApplicationScoped.
Transaction is managed by container (JTA), no extended transactions at all,
everything we do with persistence is through the entity manager, which is
@PersistenceContext(unitName = "travelcPU")
private EntityManager entityManager;
I have isolated one production server, and waited until all sessions are
dead and then flushed all caches and then I took a heapdump. It's still
1.6Gb, even with no information, I think it's quite big.
I can see in the heapdump some 22K
org.apache.openejb.persistence.JtaEntityManager . And then (And I'm not sure
if it's related to JtaEntityManager) a lot of QueryBasedValueHolder which
are in fact holding references to a lot of our domain objects.
So, my question is, shouldn't those JtaEntityManager be cleared on GC? I
think that they don't hold references to the eclipselink EnityManager, so
the problem probably comes from another part, but can you confirm?
Thanks in advance.
--
Sent from: http://tomee-openejb.979440.n4.nabble.com/TomEE-Users-
f979441.html

[cocorossello]

My xmx is 11g right now, I need it this way or production environment will
end up doing full gc. The heapdump is taken after a GC.

I think the problem comes from eclipselink. We have some caches. Those
caches are storing some entities which are in fact storing lazy relations.

In case of a batchfetch (or joinfetch), eclipselink needs to store the
original query that was performed to get the result. I guess that the
problem comes from this, as I can see a lot of QueryBasedValueHolder
objects....






--
Sent from: http://tomee-openejb.979440.n4.nabble.com/TomEE-Users-f979441.html

[Romain Manni-Bucau]

With Eclipselink take care it can trigger some unexpected connection if you
use without willing it lazy fetching (in a view for instance). This is an
easy way to leak :(.


Romain Manni-Bucau
@rmannibucau <https://twitter.com/rmannibucau> | Blog
<https://rmannibucau.metawerx.net/> | Old Blog
<http://rmannibucau.wordpress.com> | Github <https://github.com/rmannibucau> |
LinkedIn <https://www.linkedin.com/in/rmannibucau> | Book
<https://www.packtpub.com/application-development/java-ee-8-high-performance>

Post by cocorossello
My xmx is 11g right now, I need it this way or production environment will
end up doing full gc. The heapdump is taken after a GC.
I think the problem comes from eclipselink. We have some caches. Those
caches are storing some entities which are in fact storing lazy relations.
In case of a batchfetch (or joinfetch), eclipselink needs to store the
original query that was performed to get the result. I guess that the
problem comes from this, as I can see a lot of QueryBasedValueHolder
objects....
--
Sent from: http://tomee-openejb.979440.n4.nabble.com/TomEE-Users-
f979441.html

[cocorossello]

Hi,

Calling entityManager.clear() seems the best approach since most of the
queries are not executed inside a transaction. But I have no idea on where I
should use it or what would be a good implementation pattern.

I mean, let's say I have a @ViewScoped with a couple of injected
@ApplicationScoped.

@ViewScoped
public class MyViewBean{

@Inject
private MyRepo1 myRepo1;

@Inject
private MyRepo2 myRepo2;

@PostConstruct
void init(){
someVar1= myRepo1.loadSomeStuff();
someVar2 = myRepo2.loadSomeStuff();
}
..
}

@ApplicationScoped
public class MyRepo1{
@PersistenceContext(unitName = "myUnit")
private EntityManager entityManager;

public List<Something> loadSomeStuff(){ //Non transactional
entityManager.createQuery(....).getResultList();
}
}

}

So, should I call entityManager.clear() after all non transactional queries
(or at least some of them)? Can I do it in some web filter?

This seems pretty basic stuff but I can't get it right...



--
Sent from: http://tomee-openejb.979440.n4.nabble.com/TomEE-Users-f979441.html

[Romain Manni-Bucau]

tomee closes the EM after the transaction, if you need to call clear() it
means the close doesnt release the data in your provider - can be a bug.

In terms of pattern you need to clear it when you don't need it anymore. A
way to do it is a Synchronization and register it in the transaction
synchronization registry - what we do by default. Now if you don't have any
transaction it is after your last usage. The repo being not transactional
and application scoped can justify a leak, if you put it request scoped
(view scoped if you can) or @Transactional it should get fixed.


Romain Manni-Bucau
@rmannibucau <https://twitter.com/rmannibucau> | Blog
<https://rmannibucau.metawerx.net/> | Old Blog
<http://rmannibucau.wordpress.com> | Github <https://github.com/rmannibucau> |
LinkedIn <https://www.linkedin.com/in/rmannibucau> | Book
<https://www.packtpub.com/application-development/java-ee-8-high-performance>

Post by cocorossello
Hi,
Calling entityManager.clear() seems the best approach since most of the
queries are not executed inside a transaction. But I have no idea on where I
should use it or what would be a good implementation pattern.
@ApplicationScoped.
@ViewScoped
public class MyViewBean{
@Inject
private MyRepo1 myRepo1;
@Inject
private MyRepo2 myRepo2;
@PostConstruct
void init(){
someVar1= myRepo1.loadSomeStuff();
someVar2 = myRepo2.loadSomeStuff();
}
..
}
@ApplicationScoped
public class MyRepo1{
@PersistenceContext(unitName = "myUnit")
private EntityManager entityManager;
public List<Something> loadSomeStuff(){ //Non transactional
entityManager.createQuery(....).getResultList();
}
}
}
So, should I call entityManager.clear() after all non transactional queries
(or at least some of them)? Can I do it in some web filter?
This seems pretty basic stuff but I can't get it right...
--
Sent from: http://tomee-openejb.979440.n4.nabble.com/TomEE-Users-
f979441.html

[Matthew Broadhead]

i was told to inject the entitymanager like this
@ApplicationScoped
public class EntityManagerProducer {
    @PersistenceUnit(unitName = "myDb")
    private EntityManagerFactory entityManagerFactory;

    @Produces
    @Default
    @RequestScoped
    public EntityManager create() {
        return this.entityManagerFactory.createEntityManager();
    }

    public void dispose(@Disposes @Default EntityManager entityManager) {
        if (entityManager.isOpen()) {
            entityManager.close();
        }
    }
}
injecting into your example like:
@ApplicationScoped
public class MyRepo1{
    @Inject
    private EntityManager em;

    public List<Something> loadSomeStuff(){  //Non transactional
        em.createQuery(....).getResultList();
    }
}
also with JSF i alway found @PostConstruct a bit dodgy.  way better to add
<f:metadata>
     <f:viewAction action="#{myViewBean.onload}" />
</f:metadata>
then in your bean you have an onload function (no annotation necessary)
public String onload() {
    someVar1= myRepo1.loadSomeStuff();
    someVar2 = myRepo2.loadSomeStuff();
    return null;  //you can call action here if necessary
}

Post by cocorossello
Hi,
Calling entityManager.clear() seems the best approach since most of the
queries are not executed inside a transaction. But I have no idea on where I
should use it or what would be a good implementation pattern.
@ApplicationScoped.
@ViewScoped
public class MyViewBean{
@Inject
private MyRepo1 myRepo1;
@Inject
private MyRepo2 myRepo2;
@PostConstruct
void init(){
someVar1= myRepo1.loadSomeStuff();
someVar2 = myRepo2.loadSomeStuff();
}
..
}
@ApplicationScoped
public class MyRepo1{
@PersistenceContext(unitName = "myUnit")
private EntityManager entityManager;
public List<Something> loadSomeStuff(){ //Non transactional
entityManager.createQuery(....).getResultList();
}
}
}
So, should I call entityManager.clear() after all non transactional queries
(or at least some of them)? Can I do it in some web filter?
This seems pretty basic stuff but I can't get it right...
--
Sent from: http://tomee-openejb.979440.n4.nabble.com/TomEE-Users-f979441.html

[cocorossello]

Thx Chongma, but one question.

Let's say I also want to use that MyRepo1 from a @Stateless @Scheduled (so
no request context). How can I produce that same entityManager?



--
Sent from: http://tomee-openejb.979440.n4.nabble.com/TomEE-Users-f979441.html

[cocorossello]

Romain, one question. You said that if I was using it outside a transaction I
should clear myself the entityManager.

But looking at the JtaEntityManager it does nothing in that case:

public void clear() {
if (!extended && !isTransactionActive()) {
return;
}
....
}

How can I clear the entityManager?



--
Sent from: http://tomee-openejb.979440.n4.nabble.com/TomEE-Users-f979441.html

[Romain Manni-Bucau]

it was supposed to be done on leaf call on the dsl


Romain Manni-Bucau
@rmannibucau <https://twitter.com/rmannibucau> | Blog
<https://rmannibucau.metawerx.net/> | Old Blog
<http://rmannibucau.wordpress.com> | Github <https://github.com/rmannibucau> |
LinkedIn <https://www.linkedin.com/in/rmannibucau> | Book
<https://www.packtpub.com/application-development/java-ee-8-high-performance>

Post by cocorossello
Romain, one question. You said that if I was using it outside a transaction I
should clear myself the entityManager.
public void clear() {
if (!extended && !isTransactionActive()) {
return;
}
....
}
How can I clear the entityManager?
--
Sent from: http://tomee-openejb.979440.n4.nabble.com/TomEE-Users-
f979441.html

[Matthew Broadhead]

Post by cocorossello
Thx Chongma, but one question.
no request context). How can I produce that same entityManager?
--
Sent from: http://tomee-openejb.979440.n4.nabble.com/TomEE-Users-f979441.html

[Matthew Broadhead]

sorry i misread the question.  you should still be able to @Inject into
a @Stateless without a @RequestContext?
for my timer classes i used this example
http://tomee.apache.org/examples-trunk/schedule-expression/README.html

Post by cocorossello
Thx Chongma, but one question.
no request context). How can I produce that same entityManager?
--
Sent from: http://tomee-openejb.979440.n4.nabble.com/TomEE-Users-f979441.html