BaCo2V2O8 consists of Co chains in which a Co2+ ion carries a fictitious spin 12 with Ising anisotropy. We performed elastic and inelastic neutron scattering experiments in BaCo2V2O8 in a magnetic field perpendicular to the c axis which is the chain direction. With applying magnetic field along the a axis at 3.5 K, the antiferromagnetic order with the easy axis along the c axis, observed in zero magnetic field, is completely suppressed at 8 T, while the magnetic field gradually induces an antiferromagnetic order with the spin component along the b axis. We also studied magnetic excitations as a function of transverse magnetic field. The lower boundary of the spinon excitations splits gradually with increasing magnetic field. The overall feature of the magnetic excitation spectra in the magnetic field is reproduced by the theoretical calculation based on the spin 12 XXZ antiferromagnetic chain model, which predicts that the dynamic magnetic structure factor of the spin component along the chain direction is enhanced and that along the field direction has clear incommensurate correlations.