Background: Family may play an important role in the origin, maintenance, and treatment of people with social withdrawal. The aim of this study is to analyze family factors related to social withdrawal syndrome. Methods: Socio-demographic, clinical, and family data, including family psychiatric history, dysfunctional family dynamics, and history of family abuse were analyzed in 190 cases of social withdrawal with a minimum duration of 6 months that started an at-home treatment program. Data were analyzed at baseline and at 12 months. Results: In 36 cases (18%) neither the patient nor the family allowed at home evaluation and treatment by the Crisis Resolution Home Treatment (CRHT) team. Patients had high rates of dysfunctional family dynamics (n = 115, 61.5%), and family psychiatric history (n = 113, 59.3%), especially maternal affective (n = 22, 42.9%), and anxiety disorders (n = 11 20.4%). There was a non-negligible percentage of family maltreatment in childhood (n = 35, 20.7%) and single-parent families (n = 66, 37.8%). Most of the cases lived with their families (n = 135, 86%), had higher family collaboration in the therapeutic plan (n = 97, 51.9%) and families were the ones to detect patient isolation and call for help (n = 140, 73.7%). Higher social withdrawal severity (as defined by at least one of: early age of onset, no family collaboration, lack of insight, higher CGSI score, and higher Zarit score), was associated with family psychiatric history, dysfunctional family dynamics, and family abuse history. All of these predictive variables were highly correlated one to each other. Conclusions: There is a high frequency of family psychiatric history, dysfunctional family dynamics, and traumatic events in childhood (family maltreatment), and these factors are closely interrelated, highlighting the potential role of family in the development and maintenance of social withdrawal.