Critical Reflection

SGSupermarket chatbot provides the user with the various promotions available in the 5 different supermarkets (NTUC, Giant, Cold Storage, Prime and Sheng Siong) in Singapore islandwide. It was built using Microsoft's Bot Framework that provides us - developer with the structure to develop a universal bot which can be published to all different platforms (Facebook Messenger, Telegram, Slack and Skype).

How does it connect to ideas you have learned?

Throughout the whole semester of being introduced into this whole new world of "Computational Media Literacy", with the topics ranging from digital medium to machine learning, I have gain numerous insights that is outside of my academic study.

In the very first week of lecture, we were introduced to the term – digital medium and it is “new, powerful, unique and still developing its grammar”. There is a rise in the number of chatbot in recent years as people are moving towards social messaging platforms. As such, Chatbot has become a new interface for people to get information. Moving from the time where people gets to know about promotions through newspapers, to now, with my solution, they can easily get the information and even make comparisons among the different promotions available in different supermarkets. People would initially have to constantly flip around the newspaper but now these information are being laid out upfront to them via their mobile devices.

Next, we moved into the concepts of Internet and Servers – network of computers. We went through the process of development of early Internet. In my solution, I made use of Azure cloud services where I published my chatbot at. This has allowed for the connection to publish a Facebook Messenger chatbot where everyone with a Facebook account can access it. Without the network of computers that has made my chatbot solution feasible, just by having the codes in a local computer would not have allowed public access to the chatbot.

After the discussion on Intellectual property, I ponder if my solution would violate any copyrights issues as I am getting information from the supermarket’s website without their permission. As such, I went online to research about it and it was mentioned in the FairPrice’s website that “You may not modify, reproduce, copy or distribute any of the content on FairPrice On for commercial purposes without our prior consent”. With that, I believe that my solution will not infringe on their intellectual property rights as my solution is not commercialised and is not used for any monetary gain.

In addition, I have decided to use an open source language – NodeJS as it allows customisation to fit our own needs and requirements. Being an open source language, there would be a lot of sample solutions available on the Internet which would help me to pick the language at a faster pace. Furthermore, by coding in an open source language meant that people can reuse my codes and build on further. This way, I am able to contribute my part to the coding community.

Lastly, the topic on machine learning and artificial intelligent discussed on how applications can constantly “learn” new items by itself. I believe this is something that we all look forward to in the near future in most of our applications where the applications can continually improve based on the user’s response. Currently, my solution is unable to do so and I hope to be able to empower my application with such ability to continually update its product list on its own as a future enhancement of my project after more research on Machine Learning.

Analyse the affordances of digital media it uses.

We were introduced to Janet Murray’s four main affordance in week 2 and they are: Encyclopaedic, Spatial, Procedural and Participatory. SG Supermarket chatbot has a high encyclopaedic and procedural affordance, with medium participatory and medium to low spatial affordance.

Firstly, encyclopaedic affordance is defined as containing huge amount of information. SG Supermarket chatbot has a generally high encyclopaedic affordance as it provides the end user with large amount of data on the promotions available in the 5 different supermarket in Singapore. It is split into 2 main categories in which the user can view the promotions by: Promotional advertisement and promotional products. It has a generally higher encyclopaedic affordance to the end users as compared to 5 other supermarket’s online website as it allows access to a combination of all of the information available in the separate sites. However, the encyclopaedic affordance is then reduced as these data are produced during runtime by retrieving from online websites and are only temporarily stored in the variables instead of any database of the application itself.

Secondly, spatial affordance refers to the ‘space’ in the application, in which are the end users able to navigate around the application easily through the vast amount of information that the application wish to bring across. SG Supermarket chatbot has a medium to low spatial affordance as being in a conversation, it is difficult to provide clear navigation to the users on how to access the various information like how a website or mobile application would have a navigation bar to help the user find their content. However, I feel that this spatial component is especially important in any application as it plays a major role in the usability of the application. Without the ease of navigation in the search for the desired content, the large amount of information are of no value. With that, I have a strong urge to combat this issue. I added a help command that provides the user with a set of instructions and examples on how they can go about interacting with the chatbot to gather their desired information. To put it simply, users can search for promotions on SGSupermarket Chatbot which works just like how we search for information in Google. Users can simply enter “promo for milo” and the chatbot will go on to look for any offers available on products related to the search term (name of the product) – milo. The spatial affordance is further improved by the way I organise the list of promotions to be displayed to the users. Knowing that there may be quite a few items on offer, I decided to display them into a list of card views and they are sent out in the order of the respective supermarkets. In addition, the spatial affordance is highly increased when compared to the original individual website as SG Supermarket chatbot provides a summary of the promotional items in each supermarket and are neatly displayed.

Thirdly, procedural affordance is described as the ability of the computer to execute conditional actions based on a set of rules. SG Supermarket chatbot is relatively high in procedural affordance as there is an need for the chatbot to understand the message being sent by the user and return the appropriate associated information. It goes through natural language processing to be able to identify the type of promotion requested by the user – advertisement from store or product. This is accomplished by constantly training the model with the various type of utterance that maybe sent by the end users. Having known the category, the requested type of information will then be processed and sent out to the user accordingly. For example, “promo for NTUC” – is detected as a store, it returns advertisement promotions from NTUC while “promo for milo” – is detected as a product, it returns the list of items related to milo.

Lastly, participatory affordance represents the ability of the application to interact in different ways. SG Supermarket chatbot has medium participatory affordance as the users are limited to only a few ways to interact with the chatbot. The user can send message to convey the idea to the chatbot on what information they would like to get and they can also click on the ‘view details’ button to look into the details of the promotions. Looking at these restrictions, I tried to explore the sharing feature on Facebook Messenger and I included the share button to the promotion items which allows the user to share the joy with their family and friends – this in turn brings about the increases in the interaction between humans.

Overall, SG Supermarket chatbot provides a generally high interactive and immersive experience to its end users through the simulation a conversation just like a human being.

How does it make use of the new abilities of computers?

Originally, even before supermarkets build their own websites, they have their promotional advertisements being published via newspapers. With the invention of computers, followed by the creation of World Wide Web, has led to the transition of physical hard copy of information into digital ones. Now that Internet has become so widely available, people can easily search for information online as and when they need.

Having the information available online has allowed me to do web scraping to get the various information rather than storing the huge amount of data information into a database which I feel that it is unnecessary as of now. This new ability of computers – web scraping has made it easy to extract the essential information using HTML from websites.

Finally, artificial intelligence is one of the new technology that builds into applications, for it to react like human being. The ability of the computer to build a model to provide natural language processing has allowed the chatbot to converse just like a human. SG Supermarket chatbot utilise this new ability of the computer that interact with the end users just like any other human being.

How does it build off other projects that you have seen?

How I get this idea?

My brother had created a SG Supermarket Android Mobile Application quite some time back. When I was using it, I realised there are still some potential features that could be added to bring the application to its fullest to provide the best user experience for its end users.

Apart from just showing the advertisement promotions, I feel that it would be great if it allows the user to search for any promotional products.

As such, I wanted to revamp the application by creating a chatbot version of it as messaging platform has been popular among the general public. Moreover, building on these messaging platform as a chatbot does not require them to download an additional application which will get people more incline to use it.

How did I build it?

It was built based on the open source Microsoft Bot Framework that has already laid out the structure to create a universal bot. There are quite a number samples provided in their GitHub which I used to build off at the start of my project – basic set up to listen for messages and sending a static message reply. I made use of the JSON response from a web link that my brother have generated from his server for the Android Mobile Application. This has allowed me to get the list of promotional advertisement without having to scrape from the various websites again.

Overcoming challenges

Throughout the whole 6 weeks of development, I have faced a number of challenges and thankfully I managed to solve most of them.

Below are some of my major challenges and how I went about solving them:

My initial idea of using Microsoft Bot Framework instead of coding the chatbot using the respective API e.g. Facebook Messenger and Telegram, was because I wanted to create a universal bot that I can publish to the different messaging platforms. However, as I work on the project, I realised the limitation of using it. It provides only the minimal structure that all the platforms share in common. Hence I have to place in additional codes to add functionalities that are specific to the respective platform. This was when I wanted to allow the user to share the promotions on the Facebook Messenger. Luckily, Microsoft Bot Framework has “channeldata” that I can add to send platform specific messages. I realised that such cross platform structure may not always be the best solution as compared to native platforms. In fact, coding in native platforms may make the codes clearer and more secure as it is directed to the specific platform without having an extra layer on top.

On top of that, as Microsoft Bot Builder SDK 4 is new, there are really very little resources online that are useful whenever I face any issue with the codes. Often, I would look at the solution available for SDK 3 and try to understand the concept behind it. This allows me find out the relevant keyword to further look for a solution for SDK 4. I also do spend some of the time, searching through the Github’s issues to look for any similar issues.

In addition, I am new to the language – Node JS and there are a lot of time where I am stuck not being able to get the desired output. This is because Node JS runs its threads asynchronously which is very different from the language – Java that I am familiar with. A lot of time, my codes returns the value before the logic is being processed. I went on to research and took some time to grasp the concept of “async”/”await”.

Furthermore, originally, I wanted to use OCR (Optical Character Recognition) to get the individual promotional items, through the use of promotional advertisement images. However, the idea was not feasible as there is a need to recognise the position of each product and its information. Given my limited knowledge on OCR, it took me quite some time to experiment it, especially with the need to create my own model and train it. With that, I approached another method instead - scrape information from the online supermarket website.

Not to mention, I am also new to LUIS (Language Understanding Intelligent Service). It was a major challenge for me to understand some of the terms and try to incorporate it into my codes to build natural language understanding to my chatbot. I was worried that I might not be able to get it to work as intended as after many tries, I still did not managed to get the intent. I went through numerous online tutorial and documentation to understand the concepts of how LUIS works.

Critical Analysis for my solution

Indeed, my solution here is not yet perfect. If I have more time, I hope to be able to increase the participatory affordance by allowing user can send their location and the chatbot will return the promotions available near the user. This way, the end users have more ways to interact with my chatbot and get meaningful information out of it. On top of it, to provide better analysis for our users, instead of just returning all of the related promotional product, I can provide a quick summary on the cheapest items and also based it off the user's preferred products too using Machine Learning.

When researching for other similar solutions, I believe being able to retrieve promotions by product name is my unique selling point as most other solutions are only able to show the promotions that the supermarket have on advertisement.

Conclusion

Overall, I am glad to have been able to achieve the desired outcome for the project. The learning curve was really steep with most of the things being alien to me at the very beginning. In addition, of all documentation types, I decided to choose to place my documentation on GitHub. This is the first time that I deal with markdown languages. Even though, it is not as challenging as picking up a brand new programming language, but I am glad it turned out well. This learning experience was exceptionally invaluable to me as I managed to create an application that is useful from scratch and this project has allowed me to show my thorough understanding of this module by applying the concepts of computational media where possible.