Well, that's about what he expected. Fine; so be it.

Enjolras doesn't bother to say that Grantaire should speak up if he has any ideas, or notices anything. With any of their other friends it would be unnecessary. With Grantaire, differently so. He'll speak up, or he won't, but inviting him to will get no helpful response.

(Enjolras is not, in fact, currently annoyed with Grantaire. He came along to help, and he's more or less on his best behavior right now. It's simply that he isn't going to count on much contribution of any great use, though he'll be pleased if he gets it.)

"Help me make sure there's no door remaining here, then. If not, we'll push on."