Tan Wei Xiang | Venture Capital | AI & Portfolio Operations

Author: twx

  • What I learned from Text Mining 400 Spam Comments on my Blog using R

    What I learned from Text Mining 400 Spam Comments on my Blog using R

    Hey everyone! Welcome back to another amazing analytics post this week. If you are a frequent visitor of my blog, and somehow made a genuine comment here, you would have noticed your comment never appears. If you saw the screenshot above, out of a total of 999 comments, I have marked 400 as spam.

    I was reading through some really interesting comments on my blog and I was thinking, why not try doing some text analytics to see what are spammers most interested in talking about on my blog.

    Some simple explanation, text mining is a common way to do sentiment analysis on long lines of text which many market researchers do not want to look through. By going through specific text found in the whole data, researchers want to find out what the general public is talking about. In this instance, I want to find out what spam comments are generally being posted to my blog.

    A bit of Data Cleaning: The very manual and boring part…

    I started off by copying 400 comments and saving it inside a txt file. As my professor always said, data analytics is about 80% data cleaning and 20% analysis. I would change the 20% analysis to 19% and add 1% in terms of insights, which is what the business world truly values.

    My First Round of Analysis

    After a whole massive cleaning exercise here are the first set of results, represented in a word cloud of my top 30 most popular words in the spam.

    The most popular keyword is http… Which means people are spamming websites.

    The most popular keyword in the list of comments is http, which many websites start with (https was also likely in the list with the s being removed and recoded as http.) The second most popular keyword is urlqhttp which is probably also a website.

    In 400 posts, there were close to 8000 times http has appeared.

    In 400 posts, there were close to 8000 instances a web address has appeared, which means on average, spammers were posting 20 links to my blog. (They are probably trying to create backlinks to their website to improve their search engine rankings, which also will damage my website search engine ranking if it has too many backlinks out.) Thankfully these comments did not see the light of day.

    Site and blog were the next highest which would make sense to come out 1.5 times per comment. Things like: This is an amazing blog/site, before adding in other things.

    These links all appeared 582 times, which should be more or less safe to assume they are posted by the same poster.

    These websites were also the most frequent in the comments, in the same frequency, it is likely that a bot has been created by a poster to consistently post the same thing over and over again. (Or perhaps he is that free and did it manually.) It was nice to know that spammers on my blog is interested in reviews, trips and books, linking things, and some German place which consists of Freiheit, which means political freedom (Yes, I learned German for 4 years before.). I did not open the links as I was worried of any potential spyware.

    Okay that is enough analysis for today. If you are interested, do drop by for round 2! If the viewership is high enough, I’ll likely run another analysis on more comments in future.

    If you liked the analysis, you may like this analysis too!

    https://tanweixiang.com/what-i-learned-from-analysing-500-new-users-using-google-analytics/

    Otherwise you might want to know how to put analytics and management together!

    Integrating Analytics and Management: Where and How to Start?
  • Reflections from 3 Months of Distance Learning on Coursera

    Reflections from 3 Months of Distance Learning on Coursera

    It has been a few months since graduation. Due to the onset of the COVID-19 Pandemic, I was unable to have my graduation trip. With my job offer just recently attained back then, I needed to find some time for productivity. It was also during this period where NTU had a new collaboration with Coursera and I decided to give it a try. Here are some of my insights and reflections. This post is not going to be so much on the content which I learned, but my general views on online learning.

    Opportunity to Learn Interesting New Content

    Less the Chinese course which I took as a quick refresher, I took a total of 9 courses on Coursera from 4 Universities (HEC Paris, IE Business School, John Hopkins University and Yonsei University), spanning topics in creativity, innovative management, marketing, retail management, data science and Korean.

    Some of these topics were not readily available in my home university at NTU, or just really hard to get a spot in as I have already exceeded the unrestricted electives requirements by a lot and I would be last to get the spots.

    By signing up for courses through distance learning, I gained exposure in more areas than I would have done in university.

    Time Management in the Absence of Formalised Assessments

    Being online courses which were paid for by the NTU or Coursera, there was not really a lot on the line for me. I had the opportunity to learn about various topics in my own time. While that was a really nice incentive, it was also a double-edged sword in the sense that there was no push to complete as many courses as I could. I had initially signed up for 16 courses, which was a really ambitious attempt.

    It was not too hard to keep pace at first when I was still at home, waiting to commence working life. However, as work commitments started to pour in, the absence of an actual space to learn and formalised assessments gave rise to procrastination and ultimately the completion of less courses. (Completed 3 courses while in the working world!)

    One way to manage was to ensure that I set aside 5-6 hours per week on the courses, which I managed to set into my weekly calendar and this taught me about time management, especially in the working world, where we no longer have the luxury of just taking courses and acing our examinations.

    Distance Learning as a Possible Format to Learn in Future

    Learning online also let me think about distance learning as a potential format to learn in future. If there is one thing the pandemic this year has done, it brought the world closer together. In the short span of 9 months, I had the chance to experience the teaching format of 4 Universities in 3 Continents. This gave rise to the potential exchange of cultural views through various peer reviewed projects, and we had heated debates through our different world views.

    Ultimately, distance learning has made me realise the importance of lifelong learning. There is so much knowledge in the world which I do not actually know. With this, let’s all prospect on as lifelong learners!

    If you liked this post, do read the follwing post on the skills required for the digitally transformed economy:

    3 Key Skills for the Digitally Transformed Economy
  • 3 Reasons: Business Analytics at Nanyang Business School

    3 Reasons: Business Analytics at Nanyang Business School

    Having recently graduated in Business Analytics, here are 3 reasons why I picked Business Analytics:

    1. Management Science and Problem Solving Skills

    My biggest takeaway from doing a business analytics degree was that merely knowing how to code in R, Python, SQL, etc does not make you a good business analytics student. Instead, understanding the problem forming and solving framework is key in tackling any business problems which we want to solve. All my modules consisted of heavily hands on projects for me to exercise good business sense, along with a sprinkle of technical and statistical flavour.

    Ultimately, in order to succeed in Business Analytics, it is not about how good your algorithm is, but how your proposed solution solves the problem at hand! While coding is a must-know, it is definitely not the crux of business analytics.

    You may be interested in this article if you would like to get started on bridging between analytics and management.

    2. Classroom Diversity and Versatility

    Business analytics gave me the chance to meet classmates with a variety of interests. I enrolled into business analytics to make an impact in the marketing and management sectors and adding value through analytics. While going through the Business Analytics curriculum, I managed to also embark on Human Resource projects, as well as a real-life business analytics consulting project with Aon. Do stay tuned in future for updates.

    I had the opportunity to interact with friends working in various sectors, including Finance, Supply Chain, Logistics, Consulting, Data Science. Previously, I had internships in Market Intelligence, Digital Marketing, Product Marketing and Human Resources in the Automotive, Information Technology and Medical Devices sector. Currently, I’m putting my knowledge to the test in the retail sector!

    Hence, you can see that the beauty of business analytics is that it can be used anywhere!

    In the meantime, here is a post on how to maximise your experience in Business School!

    3. Relevance of Business Analytics in Industry 4.0

    Initially, I took business analytics to future proof myself. All around the world, we hear buzzwords like Industry 4.0, or big data being the next big thing. Especially in Singapore, there is an increased emphasis on Technology and Analytics. All the universities in Singapore have started offering analytics as part of their degree programme offerings. In NBS, the analytics cohort in 2018 (Based on the database classes, our core module) was 4 classes, in 2020 it has almost doubled to 7 classes. Therefore, we can see a clear increase in supply of classes to meet the increasing industry demand. (Hope you like the casual economics, and fun fact some countries call business analytics econometrics!)

    Do stay tuned to some of my future blog posts on my Business Analytics curriculum review as well as other topics in the near future!

    Next, you may want to read this module selection guide if you have decided on Business Analytics! Otherwise, you may want to read this to select your General and Unrestricted Electives.

    We have also did a tiering of modules in NBS and NTU on our Youtube Channel!

    If you liked our post, do bookmark this site, or follow us on our LinkedIn page as we look forward to creating new content for you every week.

    Image credits: Photo by Luke Chesser on Unsplash

  • Integrating Analytics and Management: Where and How to Start?

    Recently, while at work, I was given the opportunity to run some data analytics on some data in order to create some business insights and recommendations for a colleague. While I have learned about Business Analytics while in school, implementing it in real life as a one-man analyst is not just a walk in the park.

    After practicing in real life and coming out with some insights so far. Here are some possibilities you could explore if you are keen on introducing data analytics into your day-to-day management work.

    1. Establish the Goal of your Project

    When given a project, there is surely an end goal which is required by whomever has assigned you the project. One way to establish the end Goal of the project is to ask the project leader who has provided the project. If he or she does not have a goal in mind, you could look into the data to propose the possibilities. With a goal, it would be easier to scope your project.

    2. Determine the Nature of the Project

    Once you have established your goals, you will have to figure out the nature of your project. Is it more descriptive in nature? Or more predictive? Do you want to see what your data says, or try to use the data to predict something else? With the nature of the project in place, it would help you to know whether you should be focusing on descriptive or predictive methods, especially since there is so many analytics tools out there and there is no way you can try everything on the same project in a limited period of time.

    3. Try your Visualisations and Models

    With so many models out there, which to use? I am also in the process of figuring this out and you could stay tuned to future blog posts if I come across the chance to do more projects.

    For now, a good example of visualisations can be through Tableau, Google Analytics and Excel Charts.

    A good example of models can be Machine Learning Models through R, Python and Microsoft Excel.

    4. Prepare Insights, Recommendations. Rinse and Repeat

    Once you are done with your models, you have to summarise your insights which are paired with specific recommendations. You can then engage your project leader, check if everything is going along the right direction. Over time, if you can continue to work on the project, find ways to consistently relook at the data, the insights and think of ways to improve the model and the connect to the business.

    At the end of the day, when you are doing analytics, always focus on the needs and requirements of the business to propose strong insights and recommendations, and not the models which you are using.

    If you’re interested in how to formulate insights, do take a look at my analysis of 500 users on Google Analytics.

    Looking to improve on your skills amidst this pandemic? Here are some skills you could learn to future proof yourself!

  • Make the Most Out of Your Business School Experience (NTU Nanyang Business School Graduate)

    Make the Most Out of Your Business School Experience (NTU Nanyang Business School Graduate)

    After attending business school for 3 years, here are some of the reflections which I have which may be beneficial to all who want to study at business school. Do note that this is an account from my own experience and many other students may experience vastly different experiences. Here are some tips from me to make the most out of your business school experience:

    1. Do as many internships as possible, across different industries, and different functions.

    While in Nanyang Business School, and many other business schools, we would have one compulsory internship which we may have to complete, I encourage you to go out of your comfort zone to experience the different industries. Some of the benefits include the following:

    Firstly, by having more internship experience, you can gain a better understanding of the world of business. Many industries operate in different styles and manners. Having experienced the automotive, information technology and medical device industries, I felt that despite having roles which were in business, in Talent Management, Market Research and Marketing, I could apply my business and analytics knowledge in a variety of ways. This allowed me a better understanding of how what I learn in school can be actively applied.

    Secondly, you have more touch points to stay as relevant as possible. You will get to understand what are the skillsets required by each industry, and you gain a head-start compared with other graduates without prior experience in the industry. An understanding of the business operations in the role would allow one to connect the dots and grow to understand more on business functions and responsibilities.

    Thirdly, culture. By having more internship experiences, you have the opportunity to compare the various kinds of cultures in each organisation. This would determine whether you believe that you fit into a particular company or industry. By seeing more types of cultures, as fresh graduates, we could also better discern what kinds of cultures fit our personal goals in developing as future business leaders.

    If you need help on looking for internships, this post may be useful:

    https://tanweixiang.com/ntu-nbs-internship-hunting-guide-tips-from-an-nbs-graduate-with-3-mnc-internship-experiences/

    2. Try to work with different people.

    While you may feel comfortable with just working with your closest pals sometimes, actively mixing it up once in a while allows the chance to tweak your working patterns to fit an ideal role.

    More often than not, when you are working with the same people across different modules, you end up fulfilling a particular role in the group, be it as the one who leads in the discussions, the contributor, task manager and so on, and this may be a missed opportunity in trying something different.

    By working with different people, you get the chance to get more creative and hone multiple skills, sometimes going just specifically on design and presentation, and other times managing extremely complex calculations. By actively switching up things, I found it an impressive learning opportunity for me.

    3. Surprise yourself by going out of your comfort zone.

    Many naysayers may say that in business school you are learning the same thing as everyone else, which means you are not really set apart from others. I would beg to differ.

    Some ways which I managed to actively surprise myself to grow my learning experience:

    • Joining over 7 communities during the course of my studies.
    • Starting a profit-making small business and collaborating managing the end-to-end of the business with both business and non-business students.
    • Going for an exchange programme to France where I had the chance to learn from many different cultures.
    • And so on…

    For me, I believe that my business school experience was a great opportunity to learn, grow, lead and serve. I hope that you would actively seek out internships, work with new people and go out of your comfort zone.

    In addition, if you liked reading about this post, do follow us on our LinkedIn Page. Also, you might know why I picked business analytics at Nanyang Business School!

    Image Credits: Original image by Wengang Zhai on Unsplash