We describe the first observation of guided acoustic-wave Brillouin scattering (GAWBS) phase noise in a digital coherent optical fiber transmission. GAWBS noise, which is a forward lightwave generated by thermally excited vibration modes in a cylindrical fiber structure, occurs coherently not only in a signal at a single carrier frequency, but also in modulated wide-band optical signals. Since the signal-to-GAWBS-noise ratio is independent of signal power, it has caused problems in various fields including quantum optics. We point out that GAWBS noise exists even in a digital coherent transmission system such as quadrature amplitude modulation (QAM) and degrades the transmission performance since the phase noise is inevitably included within the bandwidth of the transmitted data. We propose two analogue and one digital method to compensate for the GAWBS noise and demonstrate improved performance in a QAM digital coherent transmission.