Residents have little trust in their council and other agencies and feel their concerns are being ignored, a survey shows.