Many things server may be over connected and not responding.But sitting next to you may connected to the network.I have found some fix for this issue from the iPhone 4s users having trouble like this.This might be work for someone and not for others.Just try this it may work for you.
How To Fix Siri to connect to the Network:
Step 1: First Go to ‘Settings’ and then General and click Siri, and turn Siri off.
Step 2: Then Go to Settings and then click General and Go for Reset option and click “ Reset Network Settings”. Then your phone will reboots
Step 3: Again Go to Settings and General then turn beck on Siri.
Then wait for few second you Siri will connected to network.If not connected to network the do the process again with extra bit of time.
Then you may be able to start speaking with Siri again.
If you have any other fix for this issue,you can share with us.