It seems that center tapped transformer with a bridge diode is common solution to provide symmetric power supply, but we can actually do it with simpler solution, using a voltage doubler – rectifier diodes. You can see this simple circuit in the figure.

simple symmetric supply circuit schematic

The capacitor need to be large because the rectifying is half wave, so you can avoid power line hum. With the component values shown, this symmetric power supply is capable supplying 10mA at ripple voltage about 0.2V (peak to peak).  The voltage ripple can be approximated by Vripple=(20 x I)/C, where I is the current drawn by the load in mA, and C is the capacitance in uF (micro farad).