Discussion:
how to permanently delete a queue
jahlborn
2014-01-15 14:02:23 UTC
Permalink
i'm trying to figure out how to permanently delete a queue. i'm using
activemq 5.9. i purge and delete the queue and everything looks fine.
however, when i restart the server, the queue re-appears. now, this is
mostly harmless for a normal queue as it just sits there empty. however,
for a consumer queue for a virtual topic where the consumer is gone (for
good), this presents a problem. since the topic is still active, the
consumer queue will start accumulating new messages but there is no longer a
consumer removing them so the queue starts to fill up. i've seen the "gc if
inactive" feature, but in this case, the queue will not be inactive as the
underlying topic is still active. i've also seen recommendations about
setting timeouts on messages to avoid filling inactive queues, but that
seems more like a hack than a solution (not to mention that it changes the
functionality of the application). this seems like a pretty straightforward
situation (deleted queues staying deleted), so i'm confused why this seems
so complicated. failing any of this, is there some way i could audit queue
creation on startup and avoid creating queues which are empty?



--
View this message in context: http://activemq.2283324.n4.nabble.com/how-to-permanently-delete-a-queue-tp4676305.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
artnaseef
2014-01-15 18:42:50 UTC
Permalink
If I understand right, the queue is being deleted and stays removed from the
broker until the broker restarts. Is that right?

Destinations in activemq are created on-demand. With that said, there are
many ways queues can be demanded:

* Configuration in the broker's xml files
* Clients connecting and creating the queues
* Another broker in the network creating the queue

If this happens with a network of brokers, can you try isolating that broker
out of the network to see if the problem persists?

Also, try turning up debug logging on the broker's region package; that will
give some information on when destinations are being created. That's the
org.apache.activemq.broker.region package.



--
View this message in context: http://activemq.2283324.n4.nabble.com/how-to-permanently-delete-a-queue-tp4676305p4676318.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
jahlborn
2014-01-15 20:25:42 UTC
Permalink
I am using a network of brokers, but it happens when only one broker is
running. In our application, there is no static configuration, all
destinations are created on demand.

here's how i recreate the issue:

- start 2 brokers in network
- run code which creates test queue, sends some messages, receives those
messages
- stop any code which refers to the test queue
- on both brokers: purge queue from jmx console (it was already empty).
removeQueue from jmx console.
- cleanly stop both brokers
- restart one broker. during the broker.start() call, the test queue is
added to the broker (partial stack trace below). the to string of the queue
is: "queue://test2, subscriptions=0, memory=0%, size=0, in flight
groups=null 0"


at
org.apache.activemq.broker.MutableBrokerFilter.addDestination(MutableBrokerFilter.java:172)
at
org.apache.activemq.broker.region.AbstractRegion.start(AbstractRegion.java:98)
at
org.apache.activemq.broker.region.RegionBroker.start(RegionBroker.java:199)
at
org.apache.activemq.broker.jmx.ManagedRegionBroker.start(ManagedRegionBroker.java:120)
at org.apache.activemq.broker.BrokerFilter.start(BrokerFilter.java:182)
at org.apache.activemq.broker.BrokerFilter.start(BrokerFilter.java:182)
at
org.apache.activemq.broker.TransactionBroker.start(TransactionBroker.java:120)
at org.apache.activemq.broker.BrokerFilter.start(BrokerFilter.java:182)




--
View this message in context: http://activemq.2283324.n4.nabble.com/how-to-permanently-delete-a-queue-tp4676305p4676322.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
artnaseef
2014-01-15 20:49:54 UTC
Permalink
Excellent stack trace.

Looking at the code, it appears to be finding destinations from the
persistent store and creating them in memory. Let me look through it a
little more carefully to be sure.

What persistence adapter is being used?

Another question that would help to confirm what's happening: if the
persistent store is wiped between broker shutdown and startup, does the
problem go away?



--
View this message in context: http://activemq.2283324.n4.nabble.com/how-to-permanently-delete-a-queue-tp4676305p4676323.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
jahlborn
2014-01-15 20:58:51 UTC
Permalink
I'm using KahaDB. If i wipe the persistent store, then everything starts up
cleanly. it's definitely something to do with the persistent store causing
them to be revived.

i'm currently trying to see if i can come up with a simplified scenario.
for instance, if i only run a single broker, create a queue, use it, purge
it, delete it and restart, then the queue stays deleted. trying to figure
out which step makes the difference.



--
View this message in context: http://activemq.2283324.n4.nabble.com/how-to-permanently-delete-a-queue-tp4676305p4676324.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
artnaseef
2014-01-15 20:59:49 UTC
Permalink
Confirmed - that stack trace is creating destinations from the persistence
adapter.

So, if the purge and delete of the destination works properly, the
destination should be gone from the persistent store as well.

What persistent store are the brokers using?



--
View this message in context: http://activemq.2283324.n4.nabble.com/how-to-permanently-delete-a-queue-tp4676305p4676325.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
jahlborn
2014-01-15 21:05:03 UTC
Permalink
As i mentioned, the broker is using KahaDB for persistence.



--
View this message in context: http://activemq.2283324.n4.nabble.com/how-to-permanently-delete-a-queue-tp4676305p4676326.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
jahlborn
2014-01-15 21:13:00 UTC
Permalink
Okay, i managed re-create with just a single broker. i also missed a
seemingly minor step in my original description which seems to be the key
step!

1. start single broker
2. create test queue, use it, remove code using the queue
3. cleanly stop/start broker (the queue returns now, which is expected)
4. purge, delete the test queue
5. cleanly stop/start the broker (the queue returns now, which is
unexpected)

Step 3 turns out to be the _key_ step. if you do this without step 3, then
deletion "sticks". if the queue persists through a restart _before_ it is
deleted, then the deletion does not "stick".



--
View this message in context: http://activemq.2283324.n4.nabble.com/how-to-permanently-delete-a-queue-tp4676305p4676328.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
artnaseef
2014-01-15 21:19:49 UTC
Permalink
Oh, very interesting find. Let me try that.

Apologies on asking answered questions - I think we had near-simultaneous
posts.



--
View this message in context: http://activemq.2283324.n4.nabble.com/how-to-permanently-delete-a-queue-tp4676305p4676329.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
artnaseef
2014-01-15 21:24:30 UTC
Permalink
Reproduced. I'll create a Jira entry for it.

If it weren't for virtual topics, it seems likely this would have little
chance of real impact.



--
View this message in context: http://activemq.2283324.n4.nabble.com/how-to-permanently-delete-a-queue-tp4676305p4676330.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
jahlborn
2014-01-15 21:35:58 UTC
Permalink
Awesome, thanks! can you post the jira id?

agreed, an empty, unused queue is largely incidental. it becomes
problematic when attached to an active virtual topic and starts filling up!



--
View this message in context: http://activemq.2283324.n4.nabble.com/how-to-permanently-delete-a-queue-tp4676305p4676332.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
artnaseef
2014-01-15 21:37:18 UTC
Permalink
https://issues.apache.org/jira/browse/AMQ-4970




--
View this message in context: http://activemq.2283324.n4.nabble.com/how-to-permanently-delete-a-queue-tp4676305p4676333.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Loading...