non persistent messages and maxPageSize queue performance

[Alex Colomb]

I've been tracking down some performance issues on one of our queues. The queue is hosted on an embedded broker. There is one producer and one consumer. All messages sent to the queue are non-persistent. The producer initially will have a burst of ~30k messages and then slow down to 100 messages a second. During the initial burst of messages, I see the consumer slowly consuming messages and eventually a large number of messages expire off the queue before the consumer can get to them. I've tracked the issue down to the maxPageSize setting. If I leave it at the default, messages are consumed very quickly and non are expired. If I set it to 100k, I see the behavior described above.

The documentation (http://activemq.apache.org/per-destination-policies.html) describes maxPageSize as "maximum number of persistent messages to page from store at a time." Why is changing this setting affecting the delivery/consumption of non persistent messages? Why would a large value slow things down?

Here is the programmatic broker configuration:

BrokerService embeddedBroker = new BrokerService();

PolicyEntry policyEntry = new PolicyEntry();
policyEntry.setQueue( ">" );
policyEntry.setProducerFlowControl( true );
policyEntry.setAdvisoryForSlowConsumers( true );
policyEntry.setMaxPageSize( 100000 );
policyEntry.setExpireMessagesPeriod( 15000 );

PolicyMap policyMap = new PolicyMap();
policyMap.setDefaultEntry( policyEntry );
embeddedBroker.setDestinationPolicy( policyMap );
embeddedBroker.addConnector( "tcp://0.0.0.0:51515" );

I'm using Active MQ 5.10.0.

Thank you.

[Tim Bain]

Since no one responded with an explanation, I think you should submit this
as a bug in the ActiveMQ JIRA, ideally with a unit test.

Post by Alex Colomb
I've been tracking down some performance issues on one of our queues. The
queue is hosted on an embedded broker. There is one producer and one
consumer. All messages sent to the queue are non-persistent. The producer
initially will have a burst of ~30k messages and then slow down to 100
messages a second. During the initial burst of messages, I see the consumer
slowly consuming messages and eventually a large number of messages expire
off the queue before the consumer can get to them. I've tracked the issue
down to the maxPageSize setting. If I leave it at the default, messages are
consumed very quickly and non are expired. If I set it to 100k, I see the
behavior described above.
The documentation (
http://activemq.apache.org/per-destination-policies.html) describes
maxPageSize as "maximum number of persistent messages to page from store at
a time." Why is changing this setting affecting the delivery/consumption of
non persistent messages? Why would a large value slow things down?
BrokerService embeddedBroker = new BrokerService();
PolicyEntry policyEntry = new PolicyEntry();
policyEntry.setQueue( ">" );
policyEntry.setProducerFlowControl( true );
policyEntry.setAdvisoryForSlowConsumers( true );
policyEntry.setMaxPageSize( 100000 );
policyEntry.setExpireMessagesPeriod( 15000 );
PolicyMap policyMap = new PolicyMap();
policyMap.setDefaultEntry( policyEntry );
embeddedBroker.setDestinationPolicy( policyMap );
embeddedBroker.addConnector( "tcp://0.0.0.0:51515" );
I'm using Active MQ 5.10.0.
Thank you.

[artnaseef]

That is very odd. Can you create a minimal stand-alone test program that
shows the problem?

I would like to see assertions failing on the slow-consumption case and
passing on the fast-consumption case. That will make it much easier to
diagnose.



--
View this message in context: http://activemq.2283324.n4.nabble.com/non-persistent-messages-and-maxPageSize-queue-performance-tp4689365p4689748.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.