Families and disabled Londoners are being hit hardest by the lack of affordable and accessible homes being built, a report finds.