Workflow | Alan Hung

Design and Implementation of an in-House Fabric Library

Sun, 10 Dec 2017 00:00:00 +0000

Table of Contents

Abstract

This document highlights the need for a fabric library to improve efficiency in fabric-related businesses. To specify the library’s requirements, we consider three aspects: System, User, and Administrator. Based on these requirements, we define a workflow for the fabric library that includes feature extraction, machine learning, and web services. The library is implemented using the Django framework and deployed on the Google Cloud Platform. Finally, we evaluate the impact of this library, resulting in up to a 4x improvement in fabric sales and a 50% reduction in the purchase process time.

1. Introduction

As a fabric-related business, much of our daily work involves sifting through piles of fabric and communicating with team members using fabric images. This process is time-consuming, exhausting, and inefficient. For example, a designer may spend an hour in the warehouse just to find a particular color and pattern of fabric. Additionally, images may be attached to emails, social media, or even printed out during discussions among team members, making it difficult to systematically compare or review results. To address these issues, we need to create an application that can assist team members in finishing their work more efficiently.

2. Objectives

To be more specific, there are two parts in our original workflow that can be improved, as shown in Fig. 1:

Reducing the time spent on searching for fabric.
- Before: Workers need to search through all the fabric in the warehouse to find the desired one.
- After: Workers can input desired features of the fabric into the application, which will retrieve all fabrics that match the input criteria.
Streamlining the time spent on reviewing fabrics and making purchases more efficiently.
- Before: Workers and stakeholders are reviewing fabric using different tools, which can make it difficult for stakeholders to get a clear picture of all fabrics and can delay the decision-making process for purchases.
- After: Workers and stakeholders can have discussions based on the same information provided by the application. Additionally, the application’s data visualization and machine learning-based predictions can provide valuable insights into the fabric.

3. Requirement

To achieve our goals, we can break down the requirements of our application into three aspects: “System”, “User”, and “Administrator”.

System:
- Store and display information about fabrics
- Extract features from fabrics
- Use machine learning (ML) models to make sales predictions
- Provide a dashboard summarizing each order
User:
- Search and filter fabrics
- Retrieve similar fabrics
- Add fabrics to orders
Administrator:
- Evaluate results from ML models
- Batch update the database

Regarding the requirements of the System, it should be able to store fabric information and display it to users. Additionally, it needs to extract features from the fabric so that users can search, filter, and retrieve similar fabric based on those features. Furthermore, the extracted features can be used in ML models to predict sales and can also be used in a dashboard to help stakeholders or buyers have a clearer view of the order list, enabling them to make decisions more efficiently.

For User, in addition to searching for fabric, they should be able to add fabrics to their wishlist, cart, or order. As for the Administrator, metrics should be defined so that they can evaluate the ML models that make predictions on sales. Since the supplier will frequently provide new fabrics, batch updates are preferable instead of updating them one by one.

4. Workflow

The workflow can be designed logically by considering all the requirements, as shown in Fig. 2.

Bucket: To allow fabric suppliers to upload images of fabrics and serve them to our web service, we need to create a cloud storage bucket.

Image Preprocessing: Images added to the bucket need to be resized to reduce runtime in feature extraction. Additionally, converting the color space from RGB to HSV is necessary for color theme extraction.

Features Extraction: During this stage, features such as the main color, pattern size, and color theme are extracted. Furthermore, we need to obtain sales data for each fabric to use as a target for machine learning.

Machine Learning: At this stage, we perform both supervised and unsupervised learning, as described in the Implementation Details section.

Model Evaluation: When using machine learning techniques, corresponding metrics are used to evaluate the models. For example, in linear regression, the $R^2$ metric is used for accuracy assessment.

Database: The extracted features and predictions from ML models will be updated in the database.

Web Service: We use Python for the backend and HTML, JavaScript, and CSS for the frontend to create a web service.

Web Server: Since we are using Python on the backend, we can transfer HTTP requests and responses using WSGI. This can be accomplished by using Apache and its modules.

Client: Our system only requires three functions for users: searching and filtering fabrics, retrieving similar fabrics, and adding fabrics to an order.

5. Result

Our web service consists of a main page, fabric detail pages, and an order summary page, all shown in the Fig. 3.

5.1. Main Page

On the left side of the webpage, there are multiple sliders that can be used to filter the range of each feature, including color, pattern size, contrast, and so on. The web page will then display fabrics that match the selected criteria. As for the color theme, since it is a categorical feature, it is designed as a checkbox. Additionally, users can input a fabric ID to retrieve specific fabrics.

5.2. Fabric Detail Pages

The polar plot displays the color theme and range. Additionally, histograms of each channel are shown, and the percentage of each color is displayed. Quantitative features such as pattern size and contrast are also included. Finally, the top 12 options for similar fabrics are listed at the bottom of the page.

5.3 Order Summary Page

This page serves as an interactive dashboard that not only summarizes the order for the user, but also allows the user to adjust the quantity of fabric to see its impact on the order.

At the top of the page, there are two pie charts displaying the labels predicted by K-means. These charts remind the user about the proportion of each label. A higher proportion of a specific label, especially “Plain”, may be risky for sales, as discussed in the previous project.

The price and quantity of the order are displayed in the center of the page. Users can adjust the quantity using buttons provided.

Finally, the features of the fabrics are summarized using a violin plot and a pie chart located at the bottom of the page, which provide insight into the distribution of the features.

5.4 Impact

5.4.1. Efficiency

During the feature extraction stage, we implement algorithms that achieve 12x and 63x improvements in speed compared to human processing for extracting pattern size and color themes, respectively$^1$ $^2$.

When it comes to finding similar fabrics, defining the metric can be difficult due to the subjective nature of fabric images. However, the implemented algorithm has been proven to retrieve fabrics that are similar to those perceived by humans$^3$.

5.4.2. Sales

Thanks to our in-house library and the hard work of our team members, sales have increased significantly since 2018, as shown in Fig. 4. We have observed up to a 2.73 improvement compared to 2017. Additionally, we have reduced the process time for each purchase by 50%, as discussed in a previous project.

6. Conclusion

We take a systematic approach to building an in-house library for fabric. This approach includes analyzing requirements, selecting a framework, implementation, and deployment. The library has proven to be useful in significantly increasing sales and reducing the time required for the purchase and design process.

7. Appendix: Implementation Details

7.1. Framework

As Python is our main programming language, we have opted to use the Django framework to construct our web service. This framework offers all the essential components and features, including frontend development with JavaScript, HTML, and CSS, and backend interaction through an object-relational mapper (ORM).

7.2. Image Preprocessing and Feature Extraction

We use OpenCV with Python for image manipulation tasks, such as importing, exporting, resizing, and color space conversion. We also implement our own algorithm to extract features, including color themes and pattern sizes. Similar fabrics are also considered as a feature and are extracted based on perceptual similarity.

7.3. Machine learning and Model Evaluation

We perform dimensionality reduction, K-means clustering, and beta regression on the extracted features. The procedure and results are discussed in the previous project.

7.4. Database

7.4.1. Schema

Fig. 5 provides a simplified schema as an example. We have created a “Fabric” table to store the features of fabrics. For each item in an order, we create an “OrderItem” table and use “order_id” and “fabric_id” as foreign keys. Lastly, for each order, an “Order” table is created, with “user_id” used as the foreign key.

7.4.2. Batch update

To perform batch updates in a database using Django, we create a custom django-admin command. The following is an example of importing CSV data and writing it into the “Fabric” table in a database.

class Command(BaseCommand):
    help = "load fabric data"

    def handle(self, *args, **options):
        # Custom function for loading data in CSV format
        data = load_data(file="./example.csv")
				
        for i in range(1, len(data)):
            # Write data to each column
            row_data = {}
            row_data["fabric_name"] = data[i][0]
            row_data["main_color"] = data[i][1]
            ...
            # Save data to Fabric table
            item = Fabric(**row_data)
            item.save()

7.5. Web Service

We use Django’s class-based views to build our web pages. Setting up the connection with the database and template only requires few lines of code. Additionally, built-in pagination can be used to achieve multi-page content.

class ShopView(ListView):
    model = Fabric  # Table name in database
    paginate_by = 6  # 6 items for each page
    template_name = "shop/shop.html"

Meeting user requirements for filtering or searching fabric involves setting up a frontend user interface to receive user requests and a backend to query the database.

To achieve this on the frontend, a classic range slider and a checkbox can be used. On the backend, we need to use the “setup” method to receive these requests from the frontend, parse them, and make a query to the database using “get_queryset”. The following code provides an example of how to filter content in shop.html based on a user’s request for a specific hue range.

class ShopView(ListView):
    model = Fabric  # Table name in database
    paginate_by = 6  # 6 items for each page
    template_name = "shop/shop.html"
    filters = {}

    def setup(self, request, *args, **kwargs):
        """Initialize attributes shared by all view methods."""
        super().setup(request, *args, **kwargs)
        self.parse_request()

    def get_queryset(self):
        """Make queries to the database."""
        new_queryset = Fabric.objects.filter(**self.filters)
        return new_queryset
    
    def parse_request(self):
        """Receive request and update to filter for queryset"""
        low = self.request.GET.get("hue_low")
        high = self.request.GET.get("hue_high")
        self.filters["hue__range"] = [low, high]

Another requirement is to keep track of the order list across different pages. This can be achieved by using cookie in frontend. The following example demonstrates how to update a cart’s content in a cookie using JavaScript.

function updateCart(cart){
    document.cookie = "cart=" + JSON.stringify(cart)
}

function updateQuantity(fabricName, quantity, cart) {
    if (fabricName in cart) {
        cart[fabricName] = parseInt(cart[fabricName]) + parseInt(quantity)
    } else {
        cart[fabricName] = parseInt(quantity)
    }
    updateCart(cart)
}

7.6. Web Server

As our web service is built with Python, we can use WSGI to forward requests from the web server. Apache with mod_wsgi provides a simple way to set up a directory for Python scripts, and virtual environments in the configuration file. The following code provides an example:

<Macro DjangoWSGI $document $script>
    WSGIDaemonProcess demo python-home="$script/venv" python-path="$script"
    WSGIProcessGroup demo
    WSGIScriptAlias / "$script/site_settings/wsgi.py"
    <Directory "$script/site_settings">
        <Files wsgi.py>
            Require all granted
        </Files>
    </Directory>
</Macro>

7.7. Client (User Interface)

We have created three pages for our user interface based on our requirements: the main page, fabric detail pages, and order summary page, as shown in Fig. 6.

7.7.1. Main Page

We use range sliders to allow users to specify the range of each feature. A search box is used to retrieve fabric based on the fabric’s ID.

To create range sliders, begin by adding three progress bars that will serve as its appearance.

<div class="wrap-filter-bar">
    <div id="hue-filter-bar">
        <div class="hue-progress"></div>
        <div class="hue-progress-mask-left"></div>
        <div class="hue-progress-mask-right"></div>
    </div>
</div>

Next, add two inputs to specify the upper and lower bounds of the feature.

<div class="hue-input">
    <div class="field">
        <label for="hue_min">
            <input type="range" id="hue_min"...>
        </label>
        <label for="hue_max">
            <input type="range" id="hue_max"...>
        </label>
   </div>
</div>

Add an event listener to each input to update the progress bars based on the user’s input.

function slider(name, gap) {
    // Retrieve html elements
    let rangeInput = document.querySelectorAll(`.${name}-input input`);
    let progress = document.querySelector(`.leftbar #${name}-filter-bar .${name}-progress`);
    let progressMaskLeft = document.querySelector(`.leftbar #${name}-filter-bar .${name}-progress-mask-left`);
    let progressMaskRight = document.querySelector(`.leftbar #${name}-filter-bar .${name}-progress-mask-right`);

    // Add event listener
    rangeInput.forEach(input => {
        input.addEventListener("input", function (event) {
            // Retrieve value from input
            let minVal = parseInt(rangeInput[0].value)
            let maxVal = parseInt(rangeInput[1].value)

            // Update progress bar
            if (maxVal - minVal < gap) {
                if (event.target.className === `${name}-min`) {
                    rangeInput[0].value = maxVal - gap;
                } else {
                    rangeInput[1].value = minVal + gap;
                }
            } else {
                updateProgressAndDisplay(...);
            }
        });
    });
}

Finally, create a button to submit the input values to the backend.

<button type="submit">Filter</button>

A search box can be created simply by using HTML.

<div class="search-product">
    <form class="search-box" method="GET" action="">
        <label for="search-product">
            <input type="text"
                   name="search-product"
                   id="search-product"
                   placeholder="Search Products...">
        </label>
        <button type="submit"></button>
    </form>
</div>

7.7.2. Fabric Detail Pages and Order Summary

The primary goal is to visualize the features for each fabric and summarize the order with plots. To achieve this, we rely on Plotly, JavaScript, and CSS, as they provide interactive and responsive plots for users. For example, a scatter plot can be created by providing the id of the HTML element and the x and y data, respectively.

function scatterPlot(plotID, x, y) {
    let data = [
        {
            x: x,
            y: y,
            type: "scatter",
            mode: "line",
        }
    ];
    Plotly.newPlot(plotID, data)
}

7.8. Cloud provider

We chose Google Cloud Platform (GCP) as our cloud provider because it has the lowest latency of 18 ms in Taiwan compared to AWS and Azure. Although GCP provides many convenient options for deploying a web service, such as App Engine or Cloud Run, we have chosen to use Compute Engine. The reason for this is that it allows for more precise control and direct management of virtual machines (VMs). For instance, we can select CPU and RAM according to our budget, and choose a region with the lowest latency.

Workflow Optimization for Social Media Promotion

Mon, 01 Jan 0001 00:00:00 +0000

Table of Contents

Abstract

This document discusses a workflow for social media promotion, including issues with the original workflow and a revised version that integrates online collaboration, assessment and summary, fabric selection, and a library for similar products and fabric.

The revised workflow led to a 4.4x increase in the number of posts and a 3.2x increase in the number of reactions per post, all while maintaining the same operational budget as the original workflow. The document also includes a comparison of post performance with that of a competitor, as well as a selection of top-performing posts.

1. Introduction

Social media has become an essential tool for business development due to its large number of active users and its ability to provide a direct way to interact with users. This can be leveraged for brand and product promotion. Among the many social media platforms, Facebook is often chosen because it is the largest, with 2.99 billion monthly active users as of March 31, 2023.

Social media platforms provide multiple formats, including text, images, and videos, for business owners to create posts. Video, which can generate 1200% more shares than text and image content combined, is the most effective format.

Even with knowledge of which platform to choose and what content format to use, creating posts and effectively engaging users is still difficult for businesses. There are two main challenges to consider when using social media for business purposes:

The high number of active users means that there is a lot of competition, so businesses need to consistently create new and innovative posts to stand out.
As social media becomes increasingly intertwined with product development and manufacturing, the workflow for managing it becomes more complex. However, there is no single standard workflow that fits all situations.

Therefore, designing a smooth workflow that can generate innovative posts is crucial to succeeding in business development on social media.

2. Workflow

2.1. First Version

Fig. 1 provides an example of our original workflow, which includes the following steps:

Ideation, Summary and Review: Every week, all staff members meet to propose and discuss ideas. The ideas are then summarized and reviewed by stakeholders, as shown in Fig. 1(a). Ideas that pass the review process proceed to the next step.

Preprocess: In this step, the tailor creates a prototype based on the proposal, assesses manufacturing difficulties and functionality, and decides on the fabric to be used.

Production: The product is customized based on feedback and assessment of the prototype. In addition, a videographer and photographer are required to record videos and take photos, respectively.

Release and Review: After publishing the post, all team members held a meeting to discuss its metrics and determine what needs to be improved next time.

While this chronological workflow may seem effective, there are several issues:

Waste in Resources: Ideas proposed by designers may be difficult to manufacture or may require too many resources in film and photo shooting. This can lead to termination of the proposal in the later phases of the workflow, resulting in wasted resources.
Waste of Time: To solve the aforementioned problem, one might try to gather all relevant personnel during ideation, including tailors and photographers. However, this process can be time-consuming. For instance, there is no need for a photographer to join discussions on product features during brainstorming among designers. In addition, members may have different work sites, which means they need to spend extra time commuting.
Bottleneck: Since stakeholders need to manually review all proposals before moving to the next phase of the workflow, this becomes a bottleneck for the entire workflow and is prone to human error.
Inefficiency and Unreliability: As the number of products increase, it becomes difficult to identify all products that are similar to the proposed idea during the review phase. This can lead to the unintentional manufacturing of similar products without awareness.

2.2. Revised Version

The workflow has been revised to address the issues mentioned above. Three additional steps have been integrated into the workflow, as shown in Fig. 2:

Online Collaboration: We integrate our workflow with monday.com, as shown in Fig. 2(a). This platform allows all members, including tailors and photographers who are in the later phase of the process, to discuss ideas more efficiently. In this way, all members can discuss each idea in a more organized way and save time on commuting and holding meetings.

Assessment & Summary: In addition to outlining the content and purpose of the proposal, members must identify its features❶ and assess the necessary resources❷. The KNN (K-Nearest-Neighbor) algorithm is then applied to retrieve similar products from the library based on their features❸. K-means is used to prioritize each proposal based on the required resources.

Fabric Selection: Thanks to our in-house fabric library, the time required for fabric selection has been significantly reduced.

Library: We have set up a library for products similar to the fabric library. After each product release and review, members update the actual metrics and resources needed in the library. This information is then used in the Summary step.

Budget: Apart from the time spent on in-house library development and the monthly subscription fee of the online collaboration application, there are no other expenses. This means that the revised workflow can be implemented within the original operational budget.

3. Result

3.1. Dataset

Data Source: Due to confidentiality, I cannot disclose information regarding sales, customer retention, and people reached. However, I can provide a comparison of post performance based on publicly available data from Facebook. We extracted this data using a web crawler implemented with Selenium in Python.

Competitor: We have included data from our competitor, labeled “Benchmark” in the following analysis, for comparison purposes. We selected this competitor since it is the most active account on social media among all fabric retailers in Taiwan.

3.2. Comparison

As shown in Fig. 3, we have selected two metrics for comparison:

Number of Posts: This can be considered a measure of productivity.

Average Number of Reactions per Post: This metric is calculated by dividing the total number of reactions, such as “Like”, “Love”, “Haha”, “Wow”, “Sad”, and “Angry”, that users click on by the total number of posts per month. This measure can be used to gauge popularity.

To better understand the context of our development process, we divide the timeline into four phases and discuss each phase separately.

Phase I

The competitor began promoting on social media as early as 2014, three years before we did. Since then, they have consistently published posts and steadily gained popularity.

Phase II

We adopted the first version of the workflow and started promoting our product on social media. However, we encountered the issues mentioned previously, which resulted in low productivity in terms of the number of posts, and our popularity was only one-third that of our competitor.

Phase III

The revised workflow has resulted in a 4.4x increase in the number of posts compared to Phase II. This is also the first time we have surpassed our competitor. Additionally, due to the consistency in post publication and the improved quality of each post, there has been a 3.2x increase in the number of reactions per post compared to Phase II.

Phase IV

Although the revised workflow can be implemented within the original operational budget, the cost of materials will inevitably increase as more content is created and popularity becomes saturated. In other words, we are facing the problem of diminishing marginal effect.

To tackle this issue, we implemented workflow modifications (details omitted). As a result, we maintained the same level of popularity with fewer posts. Moreover, we surpassed our competitors in popularity for the first time.

3.3. Selected Posts

The following are the top three posts from our Facebook page. These posts reached 748k people with a single post, without any advertisements, and resulted in a monthly reach of 1.2M on our Facebook page.

4. Conclusion

In this document, we outline the issues with the original workflow, which include waste in resources and time, bottleneck, inefficiency, and unreliability. We present a revised workflow that integrates an online collaboration platform, an in-house library, and a machine learning algorithm to solve these issues. This workflow can be operated under the same operational budget as the previous one, while also outperforming it in productivity and post popularity, which are measured in the number of posts and average number of reactions per post.

We also discuss a comparison with our competitor. Despite the competitor starting on social media three years earlier than us, we managed to surpass them with our revised workflow in 2021.

Workflow | Alan Hung

Design and Implementation of an in-House Fabric Library

Abstract

1. Introduction

2. Objectives

The figure illustrates two scenarios: the original workflow before using the application, and the workflow after use of the application.

3. Requirement

4. Workflow

Workflow of our application

5. Result

Pages in our application.

Main Pageshow hide

Fabric Detail Pagesshow hide

Order Summary Pageshow hide

5.1. Main Page

5.2. Fabric Detail Pages

5.3 Order Summary Page

5.4 Impact

5.4.1. Efficiency

5.4.2. Sales

Fabric sales per quarter have been tracked since 2017. Since 2018, the purchase process has incorporated the in-house library, which has led to higher sales. Different colors indicate the before and after implementation.

6. Conclusion

7. Appendix: Implementation Details

7.1. Framework

7.2. Image Preprocessing and Feature Extraction

7.3. Machine learning and Model Evaluation

7.4. Database

7.4.1. Schema

The simplified schema of our database.

7.4.2. Batch update

7.5. Web Service

7.6. Web Server

7.7. Client (User Interface)

Draft layout of the user interface.

Main Pageshow hide

Fabric Detail Pagesshow hide

Order Summary Pageshow hide

7.7.1. Main Page

7.7.2. Fabric Detail Pages and Order Summary

7.8. Cloud provider

Workflow Optimization for Social Media Promotion

Abstract

1. Introduction

2. Workflow

2.1. First Version

An example of each step in our original workflow.

(a) Ideationshow hide

(b) Preprocessshow hide

(c) Productionshow hide

(d) Release & Reviewshow hide

2.2. Revised Version

Three major steps of revised workflow.

(a) Online Collaborationshow hide

(b) Assessment & Summaryshow hide

(c) Fabric Libraryshow hide

3. Result

3.1. Dataset

3.2. Comparison

The figure compares our performance with that of our competitor in two metrics: the number of posts and the average number of reactions per post.

Phase I

Phase II

Phase III

Phase IV

3.3. Selected Posts

4. Conclusion