Photodetectors have been applied to pivotal optoelectronic components of modern optical communication, sensing, and imaging systems. As a room-temperature ferroelectric van der Waals semiconductor, 2D α-In2Se3 is a promising candidate for a next-generation optoelectronic material because of its thickness-dependent direct bandgap and excellent optoelectronic performance. Previous studies of photodetectors based on α-In2Se3 have been rarely focused on the modulated relationship between the α-In2Se3 intrinsic ferroelectricity and photoresponsivity. Herein, a simple integrated process and high-performance photodetector based on an α-In2Se3/Si vertical hybrid-dimensional heterojunction was constructed. Our photodetector in the ferroelectric polarization up state accomplishes a self-powered, highly sensitive photoresponse with an on/off ratio of 4.5 × 105 and detectivity of 1.6 × 1013 Jones, and it also shows a fast response time with 43 μs. The depolarization field generated by the remanent polarization of ferroelectrics in α-In2Se3 provides a strategy for enhancement and modulation of photodetection. The negative correlation was discovered because the enhancement photoresponsivity factor of ferroelectric modulation competes with the photovoltaic behavior within the α-In2Se3/Si heterojunction. Our research highlights the great potential of the high-efficiency heterojunction photodetector for future object recognition and photoelectric imaging.