The rules of agreement for verbs of perception in the compound tenses are a bit different than for other verbs. Rather than agreeing with the direct object, as for most verbs conjugated with avoir in the compound tenses, verbs of perception only require agreement when the subject precedes the verb.
 

1) If the subject* of the infinitive precedes the verb of perception, there is agreement:

   J'ai vu tomber la fille
   I saw the girl fall
   La fille que j'ai vue tomber
   Je l'ai vue tomber

   J'ai regardé les enfants écrire
   I watched the children write
   Les enfants que j'ai regardés écrire
   Je les ai regardés écrire

   J'ai entendu arriver les étudiants
   I watched the students arrive
   Les étudiants que j'ai entendus arriver
   Je les ai entendus arriver

2) There is no agreement with the direct object* of the infinitive.

   J'ai vu les enfants écrire les lettres
Enfants is the subject, lettres is the direct object. Even if we leave out enfants, lettres is still the direct object, so there is no agreement:
   J'ai vu écrire les lettres
   I saw the letters get written
   Les lettres que j'ai vu écrire
   Je les ai vu écrire

   J'ai entendu le monsieur lire une histoire
Monsieur is the subject, histoire is the direct object:
   J'ai entendu lire une histoire
   I heard a story get read
   L'histoire que j'ai entendu lire
   Je l'ai entendu lire

   J'ai écouté une fille chanter les cantiques
Fille is the subject, cantiques is the direct object:
   J'ai écouté chanter les cantiques
   I listened to the hymns (get) sung
   Les cantiques que j'ai écouté chanter
   Je les ai écouté chanter

*How do you know whether the noun or pronoun that precedes the verb is the subject or the direct object? If it is the person or thing performing the action of the infinitive, it is the subject of the infinitive and follows agreement rule 1.

If it is not performing the action but rather is being performed on by the infinitive, it is the direct object and follows rule 2.

Page 1:Introduction to French verbs of perception
Page 2:Word order with verbs of perception
Page 3: Agreement with verbs of perception