May part of one's property be endowed (waqf) or given in charity for the benefit of one's own family?

Yes; charity and endowments that benefit needy relatives are among the best deeds, because they combine charity with keeping family ties.
When the verse "you will not attain righteousness until you spend from what you love" (Al ʿImran 3:92) was revealed, Abu Talha r.a. gave up his garden of Bairuha', and the Prophet ﷺ advised him to distribute it among his relatives (Sahih al-Bukhari 1461). In Malaysia, waqf of land or buildings is administered through the state Islamic religious council as sole trustee; there are also cash waqf and special waqf for particular purposes. Before endowing large assets, make sure the family's maintenance and needs are secure, since starting with one's own dependants comes first (Sahih al-Bukhari 1427), and talk it over with your heirs so there is no dispute later. See Managing an estate.