Fantastic Beasts is not a traditional book-to-movie adaptation. Fantastic Beasts & Where To Find Them is one of the the textbooks at Hogwarts from the Harry Potter series. The movie is the backstory of the author of Fantastic Beasts & Where To Find Them, Newt Scamander.
Since this a prequel story, you do not need to have seen or read any of the Harry Potters. I think you would enjoy (and maybe even understand) it more if you are a Harry Potter fan, but I saw Fantastic Beasts with a friend who has never seen or read any of the HPs and she really enjoyed it, too. The story does make sense and can stand alone. However, I think the future films might start diving more into the Harry Potter world, especially if/when young Dumbledore makes an appearance, and then you will probably want to know more about the Harry Potter stories.
I really enjoyed Fantastic Beasts. The special effects were pretty amazing, and I was beyond impressed by the different beasts. The beginning of the movie had a lot of set up and different storylines running, which did make it seem long but still good. I was definitely curious how all these storylines were going to be tied together in the end, but they all ended up coming together rather nicely. I thought that the characters were interesting and that the actors did a good job portraying them. Overall, Fantastic Beasts was a strong movie and I look forward to seeing where this new series will go with the next four movies.