Problems with Observers removing themselves during event

I'm working an application that controls various mechanical devices via
TCP sockets. In order to inform various objects in the application
when the mechanism state changes, I've been employing the Observer
pattern.

This has been working fine except for a specfic sceneraio. Sometimes
the event generated by the Observable causes Observers to be removed
from further event notifications, for instance if the network
connection is closed or the movement sequence is complete. In such a
case, the Observer asks to be removed from the Observable's listener
list which causes ConcurrentModificationException's. I've reproduced a
short, compilable example, that shows my dilema:

package testing;

import java.util.ArrayList;
import java.util.ListIterator;

public class Test {
    static Listener listener1;
    static Listener listener2;

    /**
     * @param args
     */
    public static void main(String[] args) {
        listener1 = new Listener("Listener #1");
        listener2 = new Listener("Listener #2");
        EventSource source = new EventSource();

        source.addListener(listener1);
        source.addListener(listener2);

        source.firePropertyChanged();

        System.out.println("Done");

    }

}

class Listener implements IEventListener{
    private String name;
    public Listener(String name){
        this.name = name;
    }

    public void propertyChanged(EventSource source) {
        source.removeListener(Listener.this);
    }

    public String getName(){
        return name;
    }

}

interface IEventListener {
    public void propertyChanged(EventSource source);
    public String getName();
}

class EventSource {
    private ArrayList<IEventListener> listeners;

    public EventSource(){
        listeners = new ArrayList<IEventListener>();
    }

    public void firePropertyChanged(){
// for (IEventListener listener : listeners){
// System.out.println("listener " + listener.getName() + ".
Collection size = " + listeners.size());
// listener.propertyChanged(this);
// System.out.println("listener " + listener.getName() + ".
Collection size = " + listeners.size());
// }

// ListIterator<IEventListener> iterator =
listeners.listIterator(listeners.size());
// while(iterator.hasPrevious()){
// IEventListener listener = iterator.previous();
// System.out.println("listener " + listener.getName() + ".
Collection size = " + listeners.size());
// listener.propertyChanged(this);
// System.out.println("listener " + listener.getName() + ".
Collection size = " + listeners.size());
// }

        ListIterator<IEventListener> iterator = listeners.listIterator();
        while (iterator.hasNext()){
            IEventListener listener = iterator.next();
            System.out.println("listener " + listener.getName() + ". Collection
size = " + listeners.size());
            listener.propertyChanged(this);
            System.out.println("listener " + listener.getName() + ". Collection
size = " + listeners.size());
        }

    }

    public void addListener(IEventListener listener){
        listeners.add(listener);
    }

    public void removeListener(IEventListener listener){
        listeners.remove(listener);
    }

}

In the firePropertyChanged method of the EventSource class, I wrote 3
different iteration loops, just to see if the method/direction of
iteration makes any difference. It does make a difference, because
using the for..in loop or the forward iterator doesn't generate an
exception, but the second listener is never notified. The problem
makes sense, since the Oberservers are causing the ArrayList of
Observable to be altered whilst in the middle of iteration. However,
I'm not sure what the best design solution is.

My first instinct is to dump a copy of the ArrayList into an Array just
prior to iterating in the firePropertyChanged method. My concern is
will whether this will scale well when thousands of events are being
generated per second. There may not be an alternative choice, but any
advice/confirmation would be appreciated.

Best regards,
Gareth