Can I setup SharePoint to support users who are external to my organisation like suppliers and/or customers, so they can access private information?

Yes, SharePoint makes it very easy to setup sites (sub-sites) that are secure but accessible to external users outside of your organisation.